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, ^ ' Abstract 

' Reliable transmission of arbitrarily correlated sources over multiple-access relay channels (MARCs) and multiple- 

access broadcast relay channels (MABRCs) is considered. In MARCs only the destination is interested in a recon- 
struction of the sources, while in MABRCs both the relay and the destination want to reconstruct the sources. In 
C/2 , addition to arbitrary correlation among the source signals at the users, both the relay and the destination are assumed 

to have side information correlated with the source signals. Sufficient conditions for reliable communication based 



on operational separation, as well as necessary conditions on the achievable source-channel rates are characterized. 
^ ' Since operational separation is generally not optimal for MARCs and MABRCs, sufficient conditions for reliable 

' communication using joint source-channel coding schemes based on a combination of the correlation preserving 

■ mapping technique with binning are also derived. For correlated sources transmitted over fading Gaussian MARCs 

. and MABRCs, conditions under which informational separation is optimal are found. 

O 
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X, 

H . I. Introduction 

The multiple-access relay channel (MARC) models a network in which several users communicate with a single 
destination with the help of a relay Examples of such a network are sensor and ad-hoc networks in which an 
intermediate relay is introduced to assist the communication from the source terminals to the destination. MARC is 
a fundamental multi-terminal channel model that generalizes both the multiple access channel (MAC) and the relay 
channel models, and has received a lot of attention in the recent years [Tl, ||2l, O, ||5l. If the relay terminal also 
wants to decode the source messages, the model is called the multiple-access broadcast relay channel (MABRC). 

While previous work on MARCs considered independent sources at the terminals, here we allow arbitrary 
correlation among the sources that should be transmitted to the destination in a lossless fashion, and also consider 
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the availability of side information correlated with the sources at the relay and the destination. The problem we 
address is to determine whether the given sources can be losslessly transmitted to the destination for a given number 
of channel uses per source sample, which is defined as the source-channel rate. 

In ||6l Shannon showed that a source can be reliably transmitted over a point-to-point channel, if and only if its 
entropy is less than the channel capacity. Hence, a simple comparison of the rates of the optimal source and channel 
codes for the respective source and channel suffices to conclude if reliable communication is feasible. This is called 
the separation theorem. The implication of the separation theorem is that independent design of source and channel 
codes is optimal. However, the optimality of source-channel separation does not generaUze to multiuser networks 
ID, ID, ifTOl . and, in general the source and channel codes needs to be jointly designed for every particular source 
and channel combination. 

The fact that the MARC generalizes both the MAC and the relay channel models reveals the difficulty of the 
problem studied here. The capacity of the relay channel, which corresponds to the special case of our problem 
with a single source terminal and no side information, remains open despite the ongoing efforts. While the capacity 
of the MAC is known for independent messages, the optimal joint source-channel code in the case of correlated 
sources is not known in general 191. Accordingly, our goal in this paper is to identify lower and upper bounds for 
the achievable source-channel rate in MARC and MABRC models. We consider only decode-and-forward (DF) 
based achievability schemes, such that the relay terminal decodes both source signals before cooperating to forward 
them to the destination. Naturally, DF-based achievable schemes for MARC directly apply to the MABRC model 
as well. Moreover, we characterize the optimal source-channel rate in some special cases. Our contributions are 
listed below. 

« We establish an operational separation based achievable source-channel rate for MARCs. The scheme is based 
on the DF scheme and uses irregular encoding, successive decoding at the relay and backward decoding at 
the destination. We show that for MARCs with correlated sources and side information, DF with irregular 
encoding yields a higher achievable source-channel rate than the rate achieved by DF with regular encoding. 
The achievability of this rate applies directly to the MABRCs. 

• We derive necessary conditions on the achievable source-channel rate of MARCs (and MABRCs). 

> We also study MARCs and MABRCs subject to independent and identically distributed (i.i.d.) fading, for both 
phase and Rayleigh fading. We find conditions for both the phase and Rayleigh fading models, under which 
informational separation is optimal. Additionally, we find conditions for the optimality of separation for fading 
MABRCs. This is the first time the optimality of separation is shown for some special MARC and MABRC 
models. Note that these models are not degraded in the sense of ifTTl . 

• We derive two joint source-channel DF-based achievability schemes for MARCs and MABRCs for a source- 
channel rate k = 1. Both proposed schemes use a combination of binning and the correlation preserving 
mapping (CPM) technique proposed in |j9l- While in the first scheme joint source-channel coding is used 
for encoding information to the relay and binning is used for encoding information to the destination; in the 
second scheme binning is used for encoding information to the relay and source-channel coding is used for 
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encoding information to the destination. A comparison between the regions achieved by the two schemes 
reveals a tradeoff: while the relay achievable region of the former is larger, the destination achievable region 
of the latter is larger To the best of our knowledge this is the first joint source-channel achievability scheme, 
based on the CPM technique, proposed for a multiuser network with a relay. 

Prior Work 

The MARC has been extensively studied from a channel coding perspective. Achievable rate regions for the 
MARC are derived in ||2l, E) and H. In ||2l Kramer et al. derive an achievable rate region for the MARC with 
independent messages. The coding scheme employed in |2| is based on decode-and-forward relaying, and uses 
regular encoding, successive decoding at the relay and backward decoding at the destination. In [31 it is shown 
that, in contrast to the classic relay channel, different DF schemes yield different rate regions for the MARC. In 
particular, backward decoding can support a larger rate region than sliding window decoding. Another DF-based 
coding scheme which uses offset encoding, successive decoding at the relay and sliding-window decoding at the 
destination is presented in ||3J- Outer bounds on the capacity region of MARCs are obtained in I?). More recently, 
the capacity regions of two classes of MARCs are characterized in 111. 

In Q Shamai and Verdu considered the availability of correlated side information at the receiver in a point-to- 
point scenario, and showed that source-channel separation still holds. The availability of receiver side information 
enables transmitting the source reliably over a channel with a smaller capacity compared to the capacity needed in the 
absence of receiver side information. In ||9l Cover et al. derived finite-letter sufficient conditions for communicating 
discrete, arbitrarily correlated sources over a multiple-access channel (MAC); however, these conditions were later 
shown by Dueck 1121 to be sufficient but not necessary. The main technique used by Cover et al. is called correlation 
preserving mapping ( CPM) in which the channel codewords are correlated with the source sequences, resulting in 
correlation between the channel inputs. CPM is extended to source coding with side information for the MAC in 
ifTSl and to broadcast channels with correlated sources in 1 14] (with a correction in ifTSll '). 

In lfT6l Tuncel distinguishes between two types of source-channel separation. Informational separation refers 
to classical separation in the Shannon sense. Operational separation refers to statistically independent source and 
channel codes that are not necessarily the optimal codes for the underlying source or the channel. Tuncel also 
shows that for broadcasting a single source to multiple receivers, each with its own side information, operational 
separation is optimal while informational separation is not. 

In lITOll Giindiiz et al. obtain necessary and sufficient conditions for the optimality of informational separation in 
MACs with correlated sources and side information at the receiver. Giindiiz et al. also obtain necessary and sufficient 
conditions for the optimality of operational separation for the compound MAC setup. Transmission of arbitrarily 
correlated sources over interference channels (ICs) is studied in iflTl . in which Salehi and Kurtas extend the CPM 
technique to ICs; however, when the sources are independent, the region derived in ITtI does not specialize to the 
Han and Kobayashi (HK) region, |18|. An achievable region based on the CPM technique which speciaUzes to the 
HK region is derived in fT9l . The case of independent sources over ICs with correlated receiver side information 
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is studied in ||20l . Liu et al. show that source-channel separation is optimal when each receiver has access to side 
information correlated with its own desired source. When each receiver has access to side information correlated with 
the interfering transmitter's source, Liu et al. provide sufficient conditions for reliable transmission based on a joint 
source-channel coding scheme using Han-Kobayashi superposition encoding and partial interference cancellation. 

Lossless transmission over a relay channel with correlated side information is studied in 11211 . Il22l . Il23l . Il25l 
and ll26l . In ll22l Giindiiz and Erkip propose a DF based achievabiUty scheme and show that operational separation 
is optimal for physically degraded relay channels as well as for cooperative relay-broadcast channels. This scheme 
is extended to multiple relay networks in ||23l . 

Prior work on source transmission over fading channels is mostly limited to the point-to-point setup (see [24] 
and references therein). However, many practical sensor network scenarios involve transmission of correlated 
sources over fading channels. We consider two types of fading models: phase fading and Rayleigh fading. Phase 
fading models apply to high-speed microwave communications where the oscillator's phase noise and the system 
timing jitter are the key impairments. Phase fading is also the major impairment in communication systems that 
employ orthogonal frequency division multiplexing Il27l . as well as in some applications of naval communications. 
Additionally, phase fading can be used to model systems which use dithering to decorrelate signals |28|. Rayleigh 
fading models are very common in wireless communications and apply to mobile communications in the presence 
of multiple scatterers without line-of-sight ||29l . The key similarity between the two fading models is the uniformly 
distributed phase of the fading process. The phase fading and Rayleigh fading models differ in the behavior of 
the fading magnitude component, which is fixed for the former but varies following a Rayleigh distribution for 
the latter. For cooperative multi-user scenarios, phase-fading models have been considered for MARCs m, 
ll30l . for broadcast-relay channels (BRCs) (|2l and for interference channels OTl . Rayleigh fading models have been 
considered for relay channels in lf32l . lf33l and ll34l . and for MARCs in [30|. 

The rest of this paper is organized as follows: in Section HIl the model and notations are presented. In Section Hill 
an operational separation-based achievable source-channel rate is presented. In Section |IV] necessary conditions on 
the achievable source-channel rates are derived. The optimality of separation for correlated sources transmitted over 
fading Gaussian MARCs is studied in Section |V] and in Section |VI] two joint source-channel achievable schemes 
are derived. Concluding remarks are provided in Section IVIII 

II. Notations and Model 

In the following we denote the set of real numbers with IH, and we use £ to denote the set of complex numbers. 
We denote random variables with upper case letters e.g. X, Y, and their realizations with lower case letters x, y. 
A discrete random variable X takes values in a set X. We use \X\ to denote the cardinality of a finite, discrete 
set X, px{x) to denote the probability mass function (p.m.f.) of a discrete RV X on X, and fx{x) to denote the 
probability density function (p.d.f.) of a continuous RV X on £. For brevity we may omit the subscript X when it is 
the uppercase version of the sample symbol x. We use Px\Y{x\y) to denote the conditional distribution of X given 
Y . We denote vectors with boldface letters, e.g. x, y; the i'th element of a vector x is denoted by Xi and we use 
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where i < j to denote {xi, Xj^i,Xj); x^ is a short form notation for x{, and unless specified otherwise 

X = x". We denote the empty set with 0, and the complement of the set B by B"^. We use H{-) to denote the 
entropy of a discrete random variable and /(•; •) to denote the mutual information between two random variables, 
as defined in ll35l ch. 2, ch. 9]. We use A*^^^\x) to denote the set of e-strongly typical sequences w.rt. distribution 
Pxix) on X, as defined in lf36l ch. 5.1], and Ae^\x) to denote the set of e-weakly typical sequences as defined 
in ll35l ch. 3]. When referring to a typical set we may omit the random variables from the notation, when these 
variables are clear from the context. We use CM{a^(j'^) to denote a proper, circularly symmetric, complex Normal 
distribution with mean a and variance fJTll , and to denote stochastic expectation. We use X — Y ~ Z io 
denote a Markov chain formed by the random variables X, y, Z as defined in ll35l ch. 2.8]. 



A. Problem Formulation for Lossless Source Coding for MARCs and MABRCs 

The MARC consists of two transmitters (sources), a receiver (destination) and a relay. Transmitter i has access 
to the source sequence 5™, for i = 1,2. The receiver is interested in the lossless reconstruction of the source 
sequences observed by the two transmitters. The relay has access to side information and the receiver has 
access to side information W"^. The objective of the relay is to help the receiver decode the source sequences. It 
is also assumed that the side information at the relay and the receiver are correlated with the source sequences. For 
the MABRC the relay is also interested in a lossless reconstruction of the source sequences. Figure [T| depicts the 
MABRC with side information setup. The MARC is obtained when the reconstruction at the relay is omitted. 



Transmitter 1 



Transmitter 2 



Relay 



y-n 
3 



MARC 

])(?/, l/3|xi,a;2,X3) 



/ cm cm \ 
A '-'1,35 '-'2,3^ 



yr 



Receiver 



Fig. 1. Multiple-access broadcast relay channel with coirelated side information. {3^^-^, S^'^) are the reconstructions of {SY^-j, S^.^) at the 
relay, and (5™, 5™) are the reconstructions at the destination. 



The sources and the side information sequences, {Si,k, S2.k, Wk, M/3,fc}feLi' arbitrarily correlated according 
to a joint distribution p{si, S2, w, W3) over a finite alphabet 5i x 1S2 x W x W3, and independent across different 
sample indices k. All nodes know this joint distribution. 

For transmission, a discrete memoryless MARC with inputs Xi, X2, X3 over finite input alphabets ^"1,^2, A3, 
and outputs Y, Y^ over finite output alphabets y, 3^3, is available. The MARC is memoryless in the sense 

^'(yfc,y3,fe|2/^"^y3,I^2;l,a;2,X3) = p{yk,y3,k\xi.k,X2.k,X3.k)- (1) 
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Definition 1. An {m,n) source-channel code for the MABRC with correlated side information consists of two 
encoding functions, 

fM:Sr^K, « = 1,2, (2) 
a set of causal encoding functions at the relay, {/s™'"''}^^!, such that 

^^■k^ f3T\yti\0^ l<k<n, (3) 

and two decoding functions 

g{m,n) . j;" X W™ ^ X 52™, (4a) 

5^"'"^ : X W3™ 5r X 52™. (4b) 
An (m, n) code for the MARC scenario is defined as in Definition [T] with the exception that the decoding function 

^(m,n) j^g^ ^^j. gjj^jgj.^ 

Definition 2. Let S™, i = 1, 2, denote the reconstruction of S*™, i — 1,2, respectively at the receiver. Let S"\,i — 
1,2, denote the reconstruction of S'™,i = 1,2, respectively at the relay. The average probability of error, Pg^™'")^ 
of an (to, n) code for the MABRC is defined as 

pim.n) A Pr ST) + (^r, 52™)} (J {(^ra- 4") ^ (^r- ^2")}}, (5) 

while for the MARC the average probability of error is defined as 

pM A Pr 1(5-^", S^) ^ (5™, S'2™)}. (6) 

Definition 3. A source-channel rate k is said to be achievable for the MABRC if, for every e > 0, there exist positive 
integers no,TOo such that for all n > nQ,m > mQ,n/m = k there exists an {m,n) code for which p^^™'"-' < e. 
The same definition applies to the MARC. 

B. Fading Gaussian MARCs 

The fading Gaussian MARC is depicted in Figure |2] In fading Gaussian MARCs, the received signals at time k 
at the receiver and the relay are given by 

Yk = Hii^kXi,k + H2i,kX2,k + H^i^kX^^k + Zk (7a) 
Ys.k — Hi3,kXi,k + H23,kX2,k + Zy,,k, (7b) 

k = 1, . . . ,n, where Z and Z^ are independent of each other, i.i.d., circularly symmetric, complex Normal RVs, 
CA/'(0, 1). The channel input signals are subject to per-symbol average power constraints: E{|Xi|^} < Pi,i — 1, 2, 3. 
In the following it is assumed that the destination knows the instantaneous channel coefficients from Transmitter 
1,1 = 1,2, and from the relay to itself, and the relay knows the instantaneous channel coefficients from both 
transmitters to itself. This is referred to as receiver channel state information (Rx-CSI). Note that the destination 
receiver does not have CSI on the links arriving at the relay, and that the relay does not have CSI on the links 
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im 



Transmitter 2 



Fig. 2. Sources transmitted over fading Gaussian MARC with side information at tlie relay and destination (additive noises are not depicted). 



arriving at the receiver It is also assumed that the sources and the relay do not know the channel coefficients on 



We consider two types of fading: phase fading and Rayleigh fading, and define the statistical model for each 
fading type as follows: 

• Phase fading channels: The channel coefficients are given by Hu^k — aue^^''-'', an e are constants 
representing the attenuation and 9;,.^ are uniformly distributed over [0, 2tt), i.i.d., and independent of each 
other and of the additive noises Z3 and Z. Due to Rx-CSI we can set an = 1. 

• Rayleigh fading channels: The channel coefficients are given by Hu^k = aiiUu^k , a;i G are constants 
representing the attenuation, and Uu^k are circularly symmetric, complex Normal RVs, Uu^k ^ CN{Q, 1), 
i.i.d., and independent of each other and of the additive noises Z3 and Z. We define JJ — : G 
{(1,1),(1,3),(2,1),(2,3),(3,1)}|. Due to Rx-CSI we can set an = 1. 



In both models the values of an are fixed and known at all users. Observe that the magnitude of the phase-fading 
process is constant, |-ffii,fc| = an, but for Rayleigh fading the fading magnitude varies between different time 
instances. 

III. Operational Separation-Based Achievable Rate for Discrete Memoryless MARCs and 

MABRCS 

In this section we derive a separation-based achievable rate for discrete memoryless (DM) MARCs and MABRCs 
with correlated sources and side information. The achievability is established by using Slepian-Wolf source coding 
||38l , and a channel coding scheme similar to the one detailed in f?, Sections II, III], which employs DF with 
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irregular block Markov encoding, successive decoding at the relay and backward decoding at the destination. 

Theorem 1. For DM MARCs and DM MABRCs with relay and receiver side information as defined in Section 
III-AI rate n is achievable if, 

H{Si\S2,W^) <>iI{Xi-Y^\X2,VuX:i) (8a) 

H[S2\Si.W^) < kI{X2-MX^,V2,X^) (8b) 

H{Si,S2\W^)< tiI{Xi , X2 ; Fa I V^i , ^^2 , Xs) (8c) 

H{Si\S2, W) < nI{Xi,X3; Y\X2,V2) (8d) 

H{S2\Si, W) < nI{X2, Xs; Y\Xi,Vi) (8e) 

H{Si,S2\W) <KliXi,X2,X3;Y), (8f) 
for an input distribution that factors as 

p{si,S2,W3,w)p{vi)p{xi\vi)p{v2)p{x2\v2)pix3\vi,V2). (9) 

Proof: The proof is given in Subsection IIII-BI ■ 
A. Discussion 

In Figure [3] we illustrate the Markov chains for the joint distribution considered in Thm. [T| 



V2 X2 

Fig. 3. Diagram of the Markov chain for the joint distribution considered in ^9). 

Remark 1. In Thm. [l] bounds ([8al)-([8cl) are constraints for decoding at the relay, while bounds ([8dli-([8f|l are 
decoding constraints at the destination. 

Remark 2. The source-channel achievable rate of Thm. [T] is established by using two different Slepian-Wolf coding 
schemes: one for the relay and one for the destination. This requires an irregular encoding scheme for the channel 
code. In regular encoding, the codebooks at the source and the relay have the same size, see for example [jSl. 
Applying regular encoding to MABRCs with correlated sources and side information leads to merging some of the 
constraints in ([8]). In particular dSal ) and dSdl i will be combined into the constraint 

ma^{H{Si\S2,W3),H{Si\S2,W)} < k min {/(Xi; ^31^2, l^i, ^3), /(^i, ^3; i^|^2, V^2)}. 
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In irregular encoding, the transmission rates to the relay and to the destination can be different, exploiting different 
quality of the side information. We conclude that for MABRCs with correlated sources and side information, irregular 
encoding yields a higher source-channel achievable rate than the one achieved by regular encoding. When the relay 
and destination have the same side information (W = W3) then the irregular and regular schemes obtain the same 
achievable source-channel rates. 

We note that when using regular encoding for MARCs, there is a single Slepian-Wolf code, hence, in the scheme 
used in Thm. 1 it is not required to recover the source sequences at the relay and the right-hand side (RHS) of the 
constraints (l8a]i-([8fll can be combined. For example, dSal i and dSdl ) will be combined into the constraint 

H{Si\S2,W) < Kmm{l{X,;Ys\X2,VuX3),I{X,,X3;Y\X2,V2)}, 
which shows that regular encoding is more restrictive for the MARC as well. 

B. Proof of Theorem |7] 

Fix a distribution p(ui)p(a;i|z;i)p(w2)p(x2|i'2)p(a;3|wi, W2). 

1) Code construction: For i ~ 1,2, assign every G 5™ to one of 2™^* bins independently according to a 
uniform distribution on Z//f = {1, 2, . . . , 2™^* }. We refer to these two bin sets as the relay bins. Denote these 
assignments by Jl . Independent from the relay bins, for « = 1,2, assign every e 5™ to one of 2'"^* bins 
independently according to a uniform distribution onUf = {1,2,..., 2™^> }. We refer to these two bin sets as the 
destination bins. Denote these assignments by ff. 

For the channel codebook, for each i = 1,2, let Rf — ^Rf and generate 2"^^" codewords Vi{uf),uf G Uf, by 
choosing the letters Vi^k{uf), k — 1,2, ... ,n, independently with distribution py. {vi_k)- Let R\ — ^R\, for every 
•Viiuf) generate 2"^^^ codewords Xi{ul,uf),u^ G U[, by choosing the letters Xi^k{ui,uf) independently with dis- 
tribution pXi\Vi{xi,k\vi^k{uf)) for all 1 < fc < n. Finally, generate one length-n relay codeword X3(uf , U2) for each 
pair {uf, U2) e UfxU2 by choosing xs^kiuf, U2) independently with distribution Px3\Vi,V2 ix3,k\vi,kiuf) , V2,kiu2)) 
for all 1 < fc < n. 

2) Encoding: (See Table H]). Consider source sequences of length Bm, sf™ G 5f™,i = 1,2. Partition each 
sequence into B length-m subsequences, b — 1,2, . . . , B. Similarly partition the corresponding side information 
sequences wf"^ and w^™ into B length-m subsequences, W3,fc, Wb, b = 1,2, . . . , B. We transmit a total of Bm 
source samples over B + 1 blocks of n channel uses each. For any fixed (m, n) with n = Km, we can achieve a 
rate arbitrarily close to k = n/m by increasing B. i.e, {B + l)n/Bm n/m = k as B ^ oo. 

In block 1, transmitter i, i = 1, 2 observes s,;.i and finds its corresponding relay bin index w[i G U^^^. It transmits 
the channel codeword Xi(w[j^,l). In block b,b = 2,...,B, source terminal i transmits the channel codeword 
Xi{ul fj,uf where G is the relay bin index for source vector 8^ ;,, and uf^_-^ G Uf is the destination 
bin index for the source vector In block B + 1, the source terminal transmits Xi{l,uf g). 

At block 6=1, the relay simply transmits X3(l, 1). Assume that at block b,b = 2, . . . , B, B + 1, the relay 
estimates (si f,_i, 82,6-1). It then finds the corresponding destination bin indices ufi,_i £ Uf , i = 1, 2, and transmits 
the channel codeword X3(u^^ b-i' ^2 6-i)- 



July 7, 201 1 



DRAFT 



10 



Node 


Block 1 


Block 2 


Block B 


Block B + 1 


User 1 


Sl,l 


SI, 2 


Sl,B 





/r(si,i) = 


/r(si,2) = "1.2 


/r{si,s) = -"^.is 


— 




/f(si,i) = <i 


/f(si,s_i)=<,B_i 


/f(si,s)=<_B 


XI K, 1,1) 


XlK,2'"l,l) 


XlK,B,«f,B_l) 




vi{l) 


VI «l) 


Vl«,s-l) 


VI «b) 


User 2 


S2,l 


S2,2 


S2,B 


_ 


/2(s2,l) = «2,1 


/2(S2,2) = «2,2 


/J(s2,b) = -"a.B 






/2^{S2,l)=4l 


/|(S2,B-1) = -"a.B-l 


/|(S2,b)=w1s 


x2K;,i,i) 


X2(m2,2'"2,i) 


■''2('"2,B' ''■2,B-l) 


X2(l,<s) 


V2(l) 


V2K1) 


V2(«2,B-l) 


V2Kb) 


Relay 


X3(l,l) 


X3(uf,i,ui,i) 


X3(^1,B-1' "2,B-l) 





TABLE I 

Encoding process for the separation based DF scheme. 



3) Decoding: The relay decodes the source sequences sequentially trying to reconstruct source blocks Si^b,i = 
1,2, at the end of channel block b as follows: assume that the relay knows (si,(,_i,S2,6-i) at the end of block 
6—1. Hence, it can find the destination bin indices {uf 6_i, ^2 6-i)- Using this information and its received signal 
y3,6, the relay channel decoder at time b decodes {u'[ ^, i,), i.e., the relay bin indices corresponding to (si,6, S2,6) 
by looking for a unique pair {u\, U2) such that: 

eA<^\X„X2,V„V2,X3,Y3). (10) 

Using the relay decoded bin indices and the side information W3,b, the relay source decoder estimates si^, S2,b- 
More precisely, given the relay bin indices ^1,^2, the relay source decoder declares (si(,,S2^) e 5™ x .S™ 
as the decoded sequences if it is the unique pair of sequences that satisfies /f (s^ (,) = Ui,f2{s2b) = ^2 
{s{ys^2,b,ws,b) & A:^"'\Si,S2,Ws). 

Decoding at the destination is done using backward decoding. The destination node waits until the end of channel 
block B+1 is received. It first tries to decode (si,b, S2,b) using the received signal at channel block B+1 and its side 
information w^. Going backwards from the last channel block to the first, we assume that the destination knows 
(si, 6+1,82,6+1) and consider decoding of (si,6,S2,6). From Si,6+ij* = 1,2, the destination knows the relay bin 
indices m[6+i- block 6 + 1 the destination channel decoder first estimates the destination bin indices {uf 6) ^2 &)' 
corresponding to Si^ based on its received signal y6+i . More precisely, the destination channel decoder looks for 
a unique pair {uf, U2) such that: 

(xiK,6+l, wf), X2(w5^6+1> ^'2)5 Vl(wf ), V2{ui),X3{uf, U^),yb+l) 

GA<''\XuX2,Vi,V2,X3,Y). (11) 
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From the decoded destination bin indices and the side information w;,, the destination source decoder estimates 
(si,6,S2.b). More precisely, given the destination bin indices (ufjuf), the destination source decoder declares 
(s^j,,S2b) e <S™ X 5™ as the decoded sequences if it is the unique pair of sequences that satisfies /('(s^j) = 
<fi{^ib) = ^2 and (sf„s^^„wb) e A:^"'\Si,S2,W). 

4) Error probability analysis: The error probability analysis is given in Appendix lAl 

IV. Necessary Conditions on the Achievable Source-Channel Rate for Discrete Memoryless 

MARCs and MABRCs 

In this section we derive upper bounds on the achievable source-channel rate for MARCs and for MABRCs with 
correlated sources and side information at the relay and at the destination. 

Theorem 2. Consider the transmission of arbitrarily correlated sources Si and 5*2 over the DM MARC with relay 
side information W3 and receiver side information W . Any achievable source-channel rate k must satisfy the 
following constraints; 

H{Si\S2,W)<nI{Xi.X^-Y\X2) (12a) 
H{S2\Si,W)<nI{X2,X3-Y\Xi) (12b) 
H{Si,S2\W)<nI{Xi.X2,X:i;Y), (12c) 
for some input distribution a;2, X3), and the constraints 

H{Si\S2,W, W^) < r, Y3\X2,V) (13a) 

H{S2\Si,W, W3) < kI{X2; Y, Fal^i, ^) (13b) 
H{Si,S2\W,W3)<Kl{Xi,X2;Y,Y3\V), (13c) 
for some input distribution p{v){xi, X2\v)p{x3\v), with |V| < 4. 

Proof: The proof is given in Subsection IIV-AI ■ 

Remark 3. The RHS expressions of the constraints in (fT3] i are similar to the broadcast bound Q when assuming that 
all relay information is available at the destination. 

Remark 4. Setting ^"2=52 = (p, constraints in ( fT2] l specializes to the converse of ll22l Thm. 3.1] for the relay 
channel. 

Theorem 3. Consider the transmission of arbitrarily correlated sources Si and S2 over the DM MABRC with relay 
side information W3 and receiver side information W . Any achievable source-channel rate k must satisfy constraints 

'Here we use the common terminology for the classic relay channel, where in the cut-set bound the cut I{X,Xi; Y) is referred to as the 
MAC bound while I{X;Y,Yi\Xi) is called the Broadcast bound |39, Ch. 17]. 
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(fr2] l as well as the following constraints: 

H{Si\S2,W3) < Kl{Xi;Y3\X2,X3) (14a) 

H{S2\Si,W3) < KliX2;Y3\X,,X3) (14b) 

H{Si,S2\W3) <KliXi,X2;Y3\X3), (14c) 

for some input distribution p{xi,X2,X3). 

Proof: The proof is given in Appendix |B] ■ 

Remark 5. In the upper bounds detailed in constraints (fT2l l and (fT4l) the input distribution is of the form p(a;i , X2, xs), 
while the input distribution of the bound detailed in constraints (fT3b is of the form p{v)p{xi,X2\v)p{x3\v). 

A. Proofs of Thm. |2] 

Let Pi"'"' ^ as 7i,m — >■ oo, for a sequence of encoders and decoders y^™'") j^™'")^ (^("i^")^ such 

that K = n/m is fixed. We will use Fano's inequahty which states 

H{ST,S^\ST. ST) < 1 + mPi™'") log \Si X S2\ 

= m5{Pj;"''''^), (15) 
where 5(.t) is a non-negative function that approaches as x 0. We also obtain 

H{ST,ST\ST,ST) > i?(5r,5^|i^",W^'",5'r,5'^) (16a) 
= (16b) 
>H{S]"\Y",W"',ST), (16c) 

where (I16al l follows from the fact that conditioning reduces entropy ll35l Thm. 2.6.5]; ( I16bb follows from the fact 
that (5™, 5™) is a function of (Y", W™) and ( |16c| i follows from non-negativity of the entropy function for discrete 
sources. 
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(17b) 
(17c) 

(17d) 



1 ) Proof of constraints (fT2] l; Constraint ( |12a| ) is a consequence of the following chain of inequalities: 

n n 

k=l k=l 

n 

= Y,[H{Yk\X2.k) - H{Yk\Xl^,Xl^,Xl^,Yl^\Y'^-^)\ (17a) 

n 

= Y^[H{Yk\X2,k) 

k=l 

-H(Yk\ST, ST , , T^™ , Xl„ Xl 1 , Xl „Yl-\Y^-') 

n 

> ^ [H{Yk I ST,W^,Y^-\X2m)- H{Yk 1 5r , ,W^,W^,Y^-^) 

k=l 
n 

= ^[H{Yk\Sq\W''\Y^-^)-H{Yk\S'^,ST,WS\W'^,Y^-^) 

k=l 

n 

fe=i 

= IiS'^,WS']Y"\ST,W"') (17e) 
> /(5r;>'"|5'2™,W^™) (17f) 

>mi?(5i|52,VK)-TO5(Fi'"'")), (17g) 

where jllaj follows from the memoryless channel assumption (see ([T]l); ( |17b| l follows from the Markov relation 
, S'J' , Wg" , VF" ) - (Xf 1 , ^2^; 1 , xl 1 , Fg';^ \Y''-^)-Yk (see EqI); (HTcll follows from the fact that conditioning 
reduces entropy; ( I17db follows from the fact that X2^k is a deterministic function of 5™; (I17el l follows from the 
chain rule for mutual information; (I17fb follows from the non-negativity of the mutual information; and (17g 
follows from the memoryless sources and side information assumption and from equations (fT5]l-(fT6]l. 



Following arguments similar to those that led to (17gi we can also show 



^/(X2,fc,X3,fc;yfe|Xi,fc) > mHiS2\Si,W)-m6iPj:"'-"'^) 

k=l 

n 

J2l{Xi,k,X2,k,X3,k;Yk) > mH{Si,S2\W)-m6{P^"''''^). 



(18a) 



(18b) 



k=l 



Using the concavity of mutual information over the set of all joint distributions p{xi,X2,X3), taking the limit as 



m,n oo and letting P, 



(7n,n) 



0, (17gi, (llSal i and ( llSbl i result in the constraints in (fTZt . 



2) Proof of constraints ( fT3] l; We begin by defining the following auxiliary random variable: 

Vk^{Y,'j\wr). fc = l,2,...,n. 



(19) 
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k=l 



k=l 



Constraint ( |13at is a consequence of the following chain of inequalities: 

n n 

HiYk,Y3,k\X,,k, X2,k,Y^^^Y\wr) 
J2[H{Yk,Y,,k\X2,k,Y,y,Wr) 



fc=i 



- H{Yk,Y3,k\Xt Xl,X^, Y,y,Y''-\W^) 

n 

> J2 [H{Yk,Y,,k\X2,k,Y^j\Y'^'\W^,W"\S^^) 



k=l 



H{Yk,Y3,k\Xt XlXl Yl-\Y''-\W^) 



= ^ [H{Yu,Y:,,k\X2M,Yl-^\Y^-\wr, , S^) 
fc=i 

- H{Yu,Y^.u\Xl Xlxl Y^^\Y''-\Wl'', , 5^, S^) 

n 

= J2 [H{Yk,Y3,k\Y,y,Y''-\WS',W^,S^) 
fe=i 

- H{Yk,Ys,k\X^, Xl Xl Yl^\Y'^-\W^, W", 5^, Sl^) 

n 

> ^ [H(Yu,Y:,^k\Yl:^\Y''-\W^, W^", 5^) 
fe=i 

- H{Yu,Y^^u\Yl^\Y^~\W^, W"\ Sl\ S^) 
= H{Y'\ Y^\W^, W"", S'i') - Y^\W^,W"',S'{', S'^) 
= liS"^; Y'',Y^\W^, W"", S^) 

> /(S'™;r"|W^3",M^",S'^) 

= HiSl^lWp, W"", S*^") - H{Sr\Y", W^, W"", S^) 

> H{Sr\W^,W"',S^)-HiSr\Y'',W"',S^) 



(20a) 



(20b) 



(20c) 



(20d) 



(20e) 



(20f) 



(20g) 

(20h) 
(20i) 



where ( I20al ) follows from ( fT9] i; ( I20bl ) follows from the fact that is a deterministic function of (Y^^^^ , W^a") 
and from the memoryless channel assumption, (see ([TJ); jlQcj follows from the fact that conditioning reduces 
entropy; jlOdj follows from causality, ||40ll ; jlOej follows from the fact that X2^k is a deterministic function of S™; 



( I20fl ) follows again from the fact that conditioning reduces entropy; ( 20g i follows from the chain rule for mutual 
information and the nonnegativity of mutual information; ilOhi follows again from the fact that conditioning reduces 
entropy; and ( I20il ) follows from the memoryless sources and side information assumption and from equations (fTsTl- 



July 7, 201 1 



DRAFT 



Following arguments similar to those that led to < |20it we can also show that 

n 

^/(X2,fc;rfe,r3,fc|^i,fc,^4) > mff(52|5i,W^,VF3) -m,5(Pi™^")) (21a) 

n 

Y,HXi,k,X2,k;Yk,Ys,k\Vk) > mH{Su S2\W,W3) - mS{P^"''''^). (21b) 

Next we introduce the time-sharing random variable Q independent of all other random variables. We have Q = k 
with probability l/n, fc G {1, 2, . . . , n}. We can write (I20il ) as 

- V /(Xi,fc; Yk,Y3,k\X2,k,Vk) = - V /(^i.,; Yq, Y3,,\X2.„ V^, Q ^ k) 
fc=i fe=i 

= 1{X^,q-Yq,Y^^q\X2,q,Vq) 

= I{Xv,Y,Y^\X2,V), (22) 

where X^ ^ Xi,q, X^ = X^^q, Y ^ Fq, Y^ ^ Fg.Q and V ^ {Vq,Q)- Since (Xi,fc,X2,fc) and ^3,^ are 
independent given 14 = (¥3*^"^, VF™), for v ~ (v, k) we have 

Pr{Xi =xi,X2 = X2,X:i = .13!^ = v} 

= Pr{Xi,fc = a;i,X2,fe = 2:2, ^3^^ ^ X'i\Vk = v,Q = k} 

= Pr{Xi,fc = a;i,X2,fe = X2\Vk ^v,Q^ fc} PrjXg.fc = x^lVk ^v,Q^k} 

= Pr{Xi = XI, X2 ^X2\V^ v}Pr{X3 = X3\V = v}. (23) 

Hence, the probability distribution is of the form given in Thm. |2] for constraints in ( fT3] ). 
From (|20ll-(|22|, we have 

F(5i|^2,H^,W^3) - <5(Pi'"'")) < «:/(Xi;y,y3|^2, V^) (24a) 
i7(52|5i, ly, W3) - <5(Pi"^"') < «:/(X2; Y, Y3\Xi,V) (24b) 
i/(5i,52|W',W^3) -'Jl/'i'"'"^) < nI{Xi,X2;Y,Y3\V). (24c) 
Finally, taking the limit as m, n — 00 and letting pj™'"-* — leads to the constraints in ( fTsT l. 

V. Optimality of Source-Channel Separation for Fading Gaussian MARCs and MABRCs 

In this section we use the separation-based achievable rate and the necessary conditions in ( fT2] i to identify 
conditions for sources transmitted over fading Gaussian MARCs with side information, under which separation is 
optimal. We begin by considering phase fading Gaussian MARCs defined in (|7J. The result is stated in the following 
theorem. 

Theorem 4. Consider the transmission of arbitrarily correlated sources 5*1 and S2 over a phase fading Gaussian 
MARC with receiver side information W and relay side information M/3. Let the channel inputs be subject to 
per-symbol power constraints 

IE{|X,|'}<P„ i = 1,2,3, (25) 
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and let the channel coefficients and the channel input powers satisfy 



(26a) 



(26b) 



al^Pi + al^P2 + flsiBj < a%Pi + 023^2- 



(26c) 



A source-channel rate n is achievable if 



H{Si\S2, W) < Klog2(l + a?iPi + a^iPs) 
H[S2\Si,W) <K\og^{l + al,P2 + al.Ps) 
H{Si,S2\W) < «log2(l + al^Pi + aj^P2 + al^P^). 



(27b) 



(27a) 



(27c) 



Conversely, if source-channel rate n is achievable, then conditions ( l27T i are satisfied with < replaced by <. 



Remark 6. The source-channel rate n in Thm. |4]is achieved by using Xi ^ CA/^(0, Pi), i £ {1, 2, 3}, all i.i.d. and 
independent of each other, and applying DF at the relay. 

Remark 7. The achievability proof of the region (O in Thm. [T] requires strong typicality. Although strong typicality 
does not apply to continuous random variables, the Markov lemma 11411 Lemma 4.1] holds for Gaussian input 
distributions, ll42l . therefore we conclude that the achievabiUty of the region (O is applicable for (l27T i as well. See 
also m Remark 30]. 

Remark 8. Note that the phase fading MARC is not degraded in the sense of ifTTI . see also ||2] Remark 33]. 

Remark 9. Remark |6] states that the source-channel rate is achieved by using Xi ^ CAf{0, Pi),i G {1,2,3} 
independent of each other; however, the achievability proof of the region in dS) uses dependent codebooks. For this 
reason, for the achievability scheme of Thm. |4] we use the channel code construction and decoding rules detailed 
in II30I Subsection 3.E], which are a special case of Thm. [T]and use independent codebooks. The decoding rules 
detailed in |[30l Subsection 3.E] imply that the destination channel decoder does not use any information provided 
by the destination source decoder The only information exchange between the source and channel coders is through 
the bin indices of the transmitted sequences delivered from the source encoders to the channel encoders and from 
the channel decoders to the source decoders. Hence, Thm. |4] implies that informational separation is optimal in this 
scenario. 

Next, we consider source transmission over Rayleigh fading MARCs. 

Theorem 5. Consider transmission of arbitrarily correlated sources Si and 52 over a Rayleigh fading Gaussian 
MARC with receiver side information W and relay side information W^. Let the channel inputs be subject to 
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per-symbol power constraints as in ( |25l l. and let the channel coefficients and the channel input powers satisfy 

1 + al^Pi + a^^Pa < ii-i (28a) 



e^^i (^) 



1 + al^P2 + a^iPa < , / (28b) 



e^^^i (^) 



1 + al,P, + a2,P2 + a^^Pa < """f' ; -y^ (28c) 

where Ei{x) = /'^^ ie^^dg, see [43, Eqn. (5.1.1)]. A source-channel rate n is achievable if 

H{Si\S2, W) < log2(l + a^ilt/nl^Pi + a^ilC/gil^Pg)} (29a) 

ff(52|5i,VK)<AtEj^{log2(l + a^i|{/2ipP2+aIi|{/3ip/'3)} (29b) 
i?(5i,52|W^)<AtE^{log2(l + a?i|C/iipPi+a^i|C/2ipF2+a2i|f/3i|2p3)}. (29c) 

Conversely, if source-channel rate k, is achievable, then conditions (l29l l are satisfied with < replaced by <. 

Proof: The proof uses lf30l Corollary 1] and follows similar arguments to those in the proof of Thm. |4] ■ 

Remark 10. The source-channel rate k, in Thm. |5]is achieved by using Xi ^ CA/^(0, Pi), i G {1, 2, 3}, all i.i.d. and 
independent of each other, and applying DF at the relay. 

A. Proof of Thm.^ 

1 ) Achievability Proof of Thm. |4} 

• Code construction: For i = 1,2, assign every e 5™ to one of 2™^' bins independently according to a 
uniform distribution over Ui = {1,2,..., 2™^*}. Denote these assignments by fi. A channel code based on 
DF with rates Ri and R2, with blocklength n, is constructed as detailed in 1301 Subsection 3.E]. Transmitter 
i can transmit 2"^' messages, thus we require Ri — j:Ri- 

• Encoding: Consider a source sequence of length Bm, sf™ G SP™,i = 1,2. Partition this sequence into B 
length-TO subsequences, Si^b, b = 1,2, . . . , B. Similarly partition the side information sequence w^"^ E VV^™^ 
into B length-rn subsequences. We transmit a total of Bm source samples over P + 1 blocks of n channel 
uses each. If we set n = Km, by increasing B we obtain a source-channel rate {B + l)n/Bm — > n/m = k 
as P ^ cxo. 

Encoding at the transmitters: In block 6, 6 = 1, 2, . . . , P, source terminal i,i — 1, 2, observes f, and finds its 
corresponding bin index f, G Ui. Each transmitter sends its corresponding bin index using the channel code 
described in ll30l Subsection 3.E]. 

Encoding at the relay: Assume that at time b the relay knows (Mi_f,_i, M2,6_i). The relay sends these bin 
indices using the channel code described in ll30l Subsection 3.E]. 

• Decoding and error probability: From ^ Thm. 9] it follows that conditions (|26] | imply that the achievable 
channel rate region for decoding at the relay contains the achievable channel rate region for decoding at the 
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destination. Hence, reliable decoding of the channel code at the destination implies reliable decoding of the 
channel code at the relay. When the channel coefficients and the channel input powers satisfy the conditions 
in (l26b , the RHS of the constraints in (|27] | characterizes the capacity region of the phase fading Gaussian 
MARC, (see ||2] Thm. 9], ll30l Subsection 3.E]). Hence, the transmitted bin indices {wi^b, U2,b}b=i can be 
reUably decoded at the destination if, 

Ri < Klog^il + aj.Pi + al.Ps) (30a) 

i?2 <Klog2(l + a^iP2 + a3iP3) (30b) 

Ri + i?2 < Klog2(l + fln^i + 021^2 + al^Ps)- (30c) 

Decoding the sources at the destination: The decoded bin indices, denoted (wi^h, U2,6)j ^ = l7 2,...,i?, are 
given to the source decoder at the destination. Using the bin indices and the side information w;,, the source 
decoder at the destination estimates (si b,S2.b). More precisely, given the bin indices (wi.fc,U2.6), it declares 
(§1,6, §2. fc) e 5*™ X S*™ to be the decoded sequences if it is the unique pair of sequences that satisfies 
/i(§i,6) = wi.fc, h{^2,b) = "2,6 and (si^f,, S2,fc, Wf,) e Al^"'^\Si, S2, W). From the Slepian-Wolf theorem ||35l 
Thm 14.4.1], (si_;,,S2,b) can be reliably decoded at the destination if 

H{Si\S2,W)<Ri (31a) 

H{S2\Si,W)<R2 (31b) 

H{Si,S2\W)<Ri+R2. (31c) 

Combining conditions ( [30] l and ( [3T1 ) yields ( |27] ) and completes the achievability proof. 
2) Converse Proof of Thm. |?} Consider the upper bound of Thm. |2] From |2, Thm. 8] it follows that for phase 
fading with Rx-CSI, the mutual information expressions on the RHS of ( fT2b are simultaneously maximized by 
Xi,X2tX^ independent, zero-mean complex Normal, Xi ^ CJ\f{0, Pi),i — 1,2,3, yielding the same expressions 
as in ( |27] i. Therefore, for phase fading MARCs, when conditions ( |26] l hold, the conditions in ( |27l ) coincide with 
those in the upper bound of ( fT2b in Thm. |2] and the conditions in ( |27] i need to be satisfied with < instead of <. 

B. Fading MABRCs 

The optimality of informational separation can be established for MABRCs using the results for the MARC with 
additional constraints, as indicated in the following theorem: 

Theorem 6. For phase fading MABRCs for which the conditions in ( l26l l hold together with 



H{Si\S2,W:i)<H{Si\S2,W) (32a) 

H{S2\SuW:i)<H{S2\SuW) (32b) 

H{Si,S2\Wz)<H{Si,S2\W), (32c) 
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the maximum achievable source-channel rate k satisfies ( |27] l. The same statement holds for Rayleigh fading 
MABRCs with ^ replacing ^ and ^ replacing ( l27T i. 

Proof: The achievability proof of Thm. |6] differs from the achievability proof of Thm. |4]only due to decoding 
the source sequences at the relay. Conditions in ( |26] | imply that the reliable decoding of the channel code at the 
destination lead to the reliable decoding of the channel code at the relay. Conditions (l32T l imply that the reliable 
decoding of the source code at the destination lead to the reliable decoding of the source code at the relay, since 
the relay achievable source rate region contains the destination achievable source rate region. Hence, if conditions 
(l26T l and ( |32] | hold, (si S2.f,) can be reliably decoded by both the relay and the destination. The converse follows 
from the upper bound of Thm. [3] and by following similar arguments to the converse proof of Thm. ID ■ 

Remark 1 1. Conditions (|32] | imply that for the scenario described in Thm. |4]regular encoding and irregular encoding 
yield the same source-channel achievable rates (see Remark |2]i; hence, the channel code construction of ll30l 
Subsection 3.E] can be used. 

VI. Joint Source-Channel Achievable Rates for Discrete Memoryless MARCs and MABRCs 

In this section we derive two sets of sufficient conditions for the achievability of source-channel rate k = 1 for 
DM MARCs and MABRCs with correlated sources and side information. Both achievability schemes are established 
by using a combination of Slepian-Wolf (SW) source coding ||38l , the CPM technique 19] and DF scheme with 
successive decoding at the relay and backward decoding at the destination. The techniques differ in the way the 
source codes are combined. In the first scheme (Thm. |7]i the SW code is used for sending information to the 
destination and CPM is used for sending information to the relay. In the second scheme (Thm. [SJ, CPM is used 
for sending information to the destination while SW code is used for sending information to the relay. 

Before presenting the results, we first motivate this section by demonstrating sub-optimahty of separate encoding 
for the MARC. 

A. Sub-Optimality of Separation for DM MARCs and MABRCs 

Consider the transmission of arbitrarily correlated sources 5*1 and 5*2 over a DM semi-orthogonal MARC (SO- 
MARC) in which the relay-destination link is orthogonal to the channel from the sources to the relay and the destina- 
tion. The SOMARC setup is characterized by the distribution chain ys, V-i\xi,X2, a^s) = p{VR\xd)p{ys: yz\xi, X2)- 
The SOMARC is depicted in Figure |4l In the following we shall show that joint source-channel code achieves sum 
rate exceeding the sum-capacity of the SOMARC. 

We begin with the outer bound on the sum-capacity of the SOMARC characterized in Proposition [T] 

Proposition 1. The sum-capacity of the SOMARC is upper bounded by 

i?i+i?2< max mm{I{Xi,X2;Y^,Ys),I{X:i-YR) + I{X^,X2;Ys)}. (33) 

p{xx)p{x2)p{x3) 



July 7, 201 1 



DRAFT 



ST- 



Transmitter 1 





Relay 






— ► 


i 


^3 





Transmitter 2 



X^ 



Fig. 4. Semi-orthogonal multiple-access relay channel. 



Receiver 



20 



Proof: From the cut-set bound, 11351 Thm. 14.10.1] it follows that 

Ri+R2< max min {/(Xi, X2; Fg, Fsl^s), /(^i, ^2, ^3; >s)} ■ 

p{xi)p{x-2)p{x3\xi,X2) 



(34) 



For the mutual information expression /(Xi, X2, X3; Yr, Ys) we derive the following upper bound 

I{XuX2,X:i-YR,Ys) = I{X:i-YR) + I{XuX2-,YR\X:i) + I{Xi,X2-,Ys\Yr) + I{X:i-Ys\X^,X (35a) 

= /(X3; Yr) + I{X,,X2; Ys\Yr) + /(X3; Ys\X,,X2, Yr) (35b) 

^I{X3;Yr)+I{X,,X2;Ys\Yr) (35c) 

<I{X3;Yr)+I{X,,X2;Ys), (35d) 

where (I35ab follows from the chain rule for mutual information; (I35bb follows from the fact that and (Xi, X2) 
are independent given X^; ( I35cb follows from the fact that I5 and X3 are independent given {Xi,X2)', and (I35db 
follows from the fact that the relay-destination link is orthogonal to the channel from the sources to the destination. 
From the orthogonality of the relay-destination link we can also conclude that 

IiXi,X2;Y3,Ys\X3) < I{Xi,X2;Y3,Ys). (36) 

Combining (1351) and (1361 ) results in (|33] |. Finally note that statistical dependence between (Xi, X2) and X3 does not 
affect the bound expressions, thus, we may set p{x3\xi, X2) = pixs) arriving to the distribution chain of Proposition 

m m 

We now consider the transmission of correlated sources {Si, S2) with the joint distribution 



p{si = 0, S2 = 0) = p{si = 0, S2 = 1) = P{S1 = 1, S2 = 1) = -, 



(37) 



over a SOMARC defined by 



Xi = Xi = X3 = y3 = yR = {o,i}, 

3^5 = {0,1, 2}, 

Yr = X3, 
Y3 = Xi®X2, 

Ys^Xi+X2. 



(38) 
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We use the same sources used in |9| to show the sub-optimality of separation over MAC. For the channel defined 
in dJST l the mutual information expression /(Xi, X2; Ya, I5) reduces to I{Xi, X2;Ys) since /(Xi, X2; Is, I5) = 
I{Xi, X2;Ys) + /(Xi, X2; YalYs), but both Y3 and Ys are deterministic functions of {Xi,X2), and Y3 is a 
deterministic function of Ys- Hence, for independent Xi and X2 

Ri+R2< max min {/(Xi, X2; Fg, i^s), /(^s; i^i?) + /(^i, ^2; ^5)} 
= max {IiXi,X2;Ys)} 

p(xi)p{x2) 

= 1.5 bits per channel use. (39) 

Here H{Si, S2) — log3 = 1.58 bits per sample. Hence, we have H{Si, S2) > I(Xi,X2; Ys), for any joint input 
distribution of the form p{xi)p{x2)- Therefore, it is not possible ,even by using Slepian-Wolf coding, to send Si 
and 5*2 reliably to Y. However, by choosing Xi = Si and X2 = S2 zero error probability is achievable. This 
example shows that separate source and channel coding is in general sub-optimal for sending arbitrarily correlated 
sources over MARCs. 

B. Joint Source-Channel Coding for MARCs and MABRCs 

Thm. I2] and Thm. [8] below present two new sets of sufficient conditions for the achievability of source-channel 
rate k = 1, obtained by combining binning and CPM. 

Theorem 7. For DM MARCs and MABRCs with relay and receiver side information as defined in Section III-AI 
source-channel rate k = 1 is achievable if, 

H{Si\S2,W^) < I{Xi-Y^\S2,X2,Vi,X^,W^) (40a) 

( ^2 1 5i , W^3 ) < / (^2 ; ^3 1 5l , ^1 , V^2 , ^3 , VK3 ) (40b) 

H{Si,S2\W:,)<I{Xi,X2;Y^\Vi,V2,X:,,W:i) (40c) 

H{Si\S2,W)<I{Xi,Xz-Y\SuX2,V2) (40d) 

H{S2\Si,W) < I{X2,X3; Y\S2,Xi, Vi) (40e) 

H{Si,S2\W) < I{Xi,X2, X3; ^2), (40f) 
for a joint distribution that factors as 

p(si, S2, wa, u;)p(wi)p(a;i \si,vi)p{v2)p{x2\s2, V2)p{x^\vi,V2)p{y3,y\xi,X2,x-i). (41) 

Proof: The proof is given in Subsection IVI-DI ■ 
Theorem 8. For DM MARCs and MABRCs with relay and receiver side information as defined in Section III-AI 
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source-channel rate k = 1 is achievable if, 

H{Si\S2,W3) < /(Xi; Fal^i, ^2, Xg) 
H{S2\Si,W3) <I{X2;Y3\S2,Xi,X3) 
HiSi, S2\W3) < I{X,,X2; YslSi, ^2, ^3) 
H{Si\S2,W)<I{Xi,X3;Y\S2,X2,W) 
H{S2\Si,W)<I{X2,X3;Y\Si,Xi,W) 
H{Si,S2\W)<IiXi,X2,X3;Y\W), 

for a joint distribution that factors as 

p{si,S2, W3, w)p{xi\si)p{x2\s2)p{x3\si,S2)piy3, y\xi,X2, X3). 

Proof: The proof is given in Subsection IVI-EI 



(42a) 
(42b) 
(42c) 
(42d) 
(42e) 
(42f) 

(43) 



C. Discussion 

In Figure |5] we illustrate the Markov chains for the joint distribution considered in Thm. [7] and Thm. [8] 





iY,Y3) 



Fig. 5. (a) Diagram of the Markov cliain for the joint distribution considered in j4U ; (b) Diagram of the Marlcov chain for the joint distribution 
considered in j43t . 



Remark 12. In Thm. [T] and Thm. |8] conditions ( I40al i-( l40cl i and ( I42al i- (l42cl l respectively, are constraints for decoding 
at the relay, while conditions ( I40dl l- (l40fb and (I42dl l- (l42fb . are decoding constraints at the destination. 

Remark 13. Consider Thm.|2l the LHS of condition ( I40al i is the residual entropy of Si when (52, W3) are known. In 
the RHS of ( I40ab . as Vi, S2, X2, X3 and W3 are given, the mutual information expression I{Xi;Y3\S2, X2,Vi,X3, W3) 
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represents the available rate that can be used to send information on the source sequence 5™. The LHS of condition 
(I40dl i is the entropy of 5*1 when (S'2, W) are known. The RHS of condition ( I40db expresses the amount of binning 
information that can be reliably transmitted from sender 1 and the relay to the destination. This follows as the mutual 
information expression in ( I40db can be written as /(Xi, X3; F jS*!, X2, V2) = /(Xi, X3; F jS"!, S'2, V2, X2, W^), 
which represents the rate that can be used to send the bin index ui, of source sequence S*™, to the destination (see 
subsection IVI-Db . This is in contrast to the decoding constraint of the relay, cf. ( I40al ). which represents the source 
information Si transmitted in excess of the bin index sent to the destination. Therefore each mutual information 
expression in ( |40a| i and ( |40d| i. represents different types of information sent by the source: either source-channel 
codeword to the relay (I40ab or bin index to the destination ( I40db . This is because SW is used for the destination 
and CPM is used for the relay. 

Consider Thm. |8] the LHS of condition ( 142 ab is the residual entropy of 5i when (S'2, W^) are known. In the 
RHS of (142 ab the mutual information expression I{Xi] Y3IS1, X2, X3) — I{Xi; YajSi, S2,X2, X3, W3) represents 
the rate that can be used to send the bin index ui, of source sequence S™ to the relay (see subsection IVI-Eb . The 
LHS of condition (I42db is the residual entropy of Si when (S2, W) are known. In the RHS of condition (I42db . as 
52, X2 and W are given, the mutual information expression /(Xi, X3; y|S2, ^2, W) represents the available rate 
that can be used to send information on the source sequence S™. 

Remark 14. For an input distribution 

p(si, S2, W3, Wl, U2, Xi,X2,X'i) = p(si, S2,Wz,w)p[vi)p{xi | Ul )p(w2 )p(a;2 | ^2 )p(a;3 I , " 2) , 

conditions ( |40] | reduce to conditions (O, and the transmission scheme reduces to a separation-based achievable 
scheme for k = 1. 

Remark 15. In both Thm. [Tjand Thm. [8] the conditions stemming from the CPM technique can be reduced to the 
MAC source channel conditions of ||9] Equation (12)]. In Thm. [7] letting Vi = V2 = X-3 = = (/), reduces the 
relay conditions in (I40ab - (l40cb to the ones in |9| with 13 as the destination. In Thm. [8] letting A'3 = W = </>, 
reduces the destination conditions in ( I40db -( l40fb to the ones in |9| with Y as the destination. 

Remark 16. Thm.|2]and Thm. |8]establish different achievability conditions. As stated in Remark[T5]and in Subsection 
IVI-AI separate source and channel coding is generally sub-optimal for sending correlated sources over DM MARCs 
and MABRCs. In Thm. [T] the CPM technique is used for sending information to the relay, while in Thm. [8] a SW 
code is used for sending information to the relay. This observation implies that the relay decoding constraints of 
Thm. I2] are looser compared to the relay decoding constraints of Thm. [8] Using similar arguments we conclude 
that the destination decoding constraints of Thm. [8] are looser compared to the destination decoding constraints of 
Thm. I2] The above observations indicate a trade-off between the two achievability schemes. 

Figure |6] depicts the cooperative relay broadcast channel (CRBC) model, which is a special case of a MARC 
with a single source terminal. For the CRBC with correlated relay and destination side information, we can identify 
exactly the optimal source-channel rate, which was previously obtained in ll22l . 
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Relay 



1,3 



ST 



Transmitter 



X3- 



Relay Channel 



Receiver 



Fig. 6. Cooperative relay broadcast cliannel with correlated side information. 



Corollary 1. For CRBC with relay and receiver side information, source-channel rate k is achievable if 

H{Si\W3)<^iI{Xi;Y3\X3) (44a) 
H{Si\W)<Kl{XuX3;Y), (44b) 

for some input distribution p{si,W3,w)p{xi, x^). Conversely, if rate k is achievable then the conditions in ( I44al i 
and ( I44bb are satisfied with < replaced by < for some input distribution p{xi,X3). 

Proof: The achievability follows from Thm. [T| by assigning X3 = V\ and S2 = Xi = V2 = (p- The converse 
follows from Thm. [3] ■ 

Remark 17. For source-channel rate k — 1, the conditions in (l44l l can also be obtained from Thm. |7]by letting 
Vi = X3, 1S2 = A2 = V2 = and considering an input distribution independent of the sources. However, if we 
consider the conditions in Thm. [8] for S2 = X2 — <j>, we obtain the following conditions for the achievability 

HiSi\W3)<IiXi;Y3\X3,Si) (45a) 
H{Si\W)<IiXi,X3;Y\W), (45b) 

for some input distribution p{si,wrj,w)p{xi\si)p{x^\si). 

Note that the RHS of inequalities in ( 145 at and ( |45b| l are smaller than those in ( |44a| l and ( |44bl l. respectively. 
Moreover, not all joint input distributions can be achieved. Hence, the conditions obtained from Thm. |8] for the 
CRBC setup are stricter than those obtained from Thm. [T] illustrating the fact that the two sets of conditions are 
not equivalent. 

D. Proof of Thm. ^ 

1) Code construction: For i — 1,2, assign every Si e S" to one of 2"^' bins independently according to a 
uniform distribution on Ui = {1, 2, . . . , 2"^'}. Denote this assignment by fi, i — 1,2. 

For the channel codebook, for each i = 1,2, generate 2"^* codewords Vi{ui),Ui G Ui, by choosing the 
letters Vi^k{ui), k = 1, 2, . . . , n, independently with distribution py. For each pair {si,Ui),i = 1,2 generate 



July 7, 201 1 



DRAFT 



25 



Node 


Block 1 


Block 2 


Block B 


Block B + 1 


User 1 


Sl.l 


Sl,2 


Sl,fl 


— 


— 


/i(si,i) = "1,1 


/i(si,s-i) = 'ii,s-i 


/i(si,s) = «i,s 


xi(si,i, 1) 


xi(si,2,ni,i) 


xi(si,B,ui,s_i) 


xi(ai, «i,s) 


vi(l) 


Vl{jll,l) 


Vi(mi,s_i) 


vi(mi,b) 


User 2 


S2,l 


S2,2 


S2,S 






/2(s2,l) = "2,1 


f2{s2,B-l) = "2,S-1 


/2(s2,s) = «2,S 


X2(S2,1' 1) 


X2{s2,2,'"2,l) 


X2(s2,B.'"2,S-l) 


X2(a2,«2,s) 


V2(l) 


V2(m2,i) 


V2(m2,S-i) 


V2(m2,s) 


Relay 


X3(l,l) 


X3(mi,1,M2,i) 


X3(mi,S-1.«2,B-i) 


X3('"l,B,M2,s) 



TABLE II 

Encoding process for the joint source-channel achievable rate of Thm.[2] 



one n-length codeword Xi{si,Ui),Si €E 5", by choosing the letters Xi,k{si,Ui) independently with distribution 
PXi\Si.Vi{^i-k\si.k,Vi.k{ui)) for all 1 < /c < n. Finally, generate one length-n relay codeword xs{ui,U2) for each 
pair (ui, U2) eUixU2 by choosing X3,a:(ui, U2) independently with distribution p^f^ i^^ y^ (2:3,*; |wi,fe(ui),W2,fe(u2)) 
for all 1 < /c < n. 

2) Encoding: (See Table HHi. Consider a source sequence sf" € Sf",i = 1,2 of length Bn. Partition this 
sequence into B length-n subsequences, Sib,b — 1,...,B. Similarly, for b = 1,2,...,B, partition the side 
information sequences w^" and lu^" into B length-n subsequences W3.h,Wh respectively. We transmit a total 
of Bn source samples over B + 1 blocks of n channel uses each. In block 1, source terminal i,i — 1, 2, transmits 
the channel codeword Xi(si,i,l). In block b,b = 2,...,B, source terminal i,i = 1,2, transmits the channel 
codeword Xi(si,f,, where G Ui is the bin index of source vector Si,h_i. In block B + 1, source 
terminal i,i = 1,2, transmits Xi{a.i,Ui,B) where G 5" is a fixed sequence. 

At block 6=1, the relay simply transmits X3(l, 1). Assume that at block b,b = 2, . . . , B, B + 1, the relay 
estimates (si,f,_i, 82,6-1). It then finds the corresponding bin indices iii.b-i G Ui,i — 1,2, and transmits the 
channel codeword X3(-ui,;,_i, -U2,&-i)- 

3) Decoding: The relay decodes the source sequences sequentially trying to reconstruct source block b, i — 1,2, 
at the end of channel block b as follows: assume that the relay knows (si,f,_i, S2,b-i) at the end of block 6 — 1. 
Hence, it can find the bin indices (iti,b_i, it2,6_i). Using this information, its received signal y^j, and the side 
information W3,;„ the relay decodes (si,f,, 82.;,), by looking for a unique pair (§1,82) such that: 

(§1, §2, Xi(si, Mi,f,_i), X2(S2, M2,6-l), Vi(ui,f,_i), V2(u2,f,-l), X3 (ui,f,_ i , U2,f,- 1 ) , W3,f, , y^^b) 

eA<''HSi,S2,Xi,X2,Vi,V2,X3,W3,Y3). (46) 

Decoding at the destination is done using backward decoding. The destination node waits until the end of 
channel block _B + 1. It first tries to decode (si,b,S2,_b) using the received signal at channel block B + 1 and its 



July 7, 201 1 



DRAFT 



26 



side information w^. Going backwards from the last channel block to the first, we assume that the destination 
knows (si.b+i, S2,6+i) and consider decoding of (si b,S2,6). At block 6 + 1 the destination channel decoder first 
estimates the destination bin indices iti ^, i — 1,2, corresponding to f, based on its received signal yb+i and the 
side information W6+i. More precisely, the destination channel decoder looks for a unique pair (ui,U2) such that: 

(Si,b+1, S2,6+l, Xi(si,b+1, Wl), X2(S2,(,+ 1, U2), Vi(mi), V2(U2), X3({ti, {t2), Wfc+i, y^+i) 

eA<''\Si,S2,Xi,X2,Vi,V2,X3,W,Y). (47) 

The decoded destination bin indices, ({ti, W2) are then given to the destination source decoder. From the destination 
bin indices and the side information w;,, the destination source decoder estimates (si f,,S2,6). More precisely, given 
the destination bin indices {■ui,U2), the destination source decoder declares (si.b,Si f,) as the decoded sequences if 
it is the unique pair of sequences that satisfies /i(si,6) = ui, /2(s2,6) — U2 and (si.b, S2.b, Wb) e A*'-"-' (5*1, 52, W). 

4) Error probability analysis and corresponding rate constraints: The error probability analysis is given in 
Appendix |C] It is shown that relay decoding (l46T l can be done reliably if (I40ab -( l40cl i hold and decoding the source 
sequences at the destination can be done reliably as long as ( |40d| i- (|40ft hold. 

E. Proof of ThmM 

1) Code construction: For i = 1,2, assign every G 5" to one of 2"^' bins independently according to a 
uniform distribution on Ui = {1, 2, . . . , 2"^'}. Denote this assignment by fi, i — 1,2. 

For i = 1,2, for each pair {ui,Si),Ui & Ui,Si G 5™, generate one n-length codeword Xi{ui,Si), by choosing 
the letters Xi^k{ui,Si) independently with distribution PXi\Sii^i,k\si,k) for all 1 < fc < n. Finally, generate one 
length-n relay codeword X3(si,S2) for each pair (81,82) G 5" x 52 by choosing X3 fc(si,S2) independently with 
distribution px3|Si,S2 (2:3, fc|si,fc,S2,fc) for all 1 < fc < n. 

2) Encoding: (See Table HiHi. Consider a source sequence sf" G SP",i = 1,2 of length Bn. Partition this 
sequence into B length-n subsequences, Si^b,b = 1,...,B. Similarly, for b = 1,2,...,B, partition the side 
information sequences and w^" into B length-n subsequences W3 b,Wb respectively. We transmit a total 
of Bn source samples over B + 1 blocks of n channel uses each. In block 1, source terminal i,i = 1, 2, observes 
Si 1 and finds its corresponding bin index 1 G Ui. It transmits the channel codeword Xi{ui,i, a^) where G 5" is a 
fixed sequence. In block b,b — 2, . . . , B, source terminal i,i — 1, 2, transmits the channel codeword Xi{uij,, 8^ f,_i) 
where b G Ui is the bin index of source vector Si,i,. In block B + 1, source terminal i,i = 1,2, transmits 

Xj(l,8i^s). 

At block 6 = 1, the relay simply transmits X3(ai, a2). Assume that at block b,b = 2, . . . , B, B + 1, the relay 
estimates = 1,2. It then transmits the channel codeword X3(si.b_i, S2.b-i). 

3) Decoding: The relay decodes the source sequences sequentially trying to reconstruct source block b, * = 1, 2, 
at the end of channel block 6 as follows: assume that the relay knows s,;.b-i, * = 1, 2, at the end of block 6—1. 
Using this information, its received signal y3 b, and the side information W3 b-i, the relay channel decoder at time 
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Node 


Block 1 


Block 2 


Block B 


Block B+1 




Sl,l 


Sl,2 


si,s 


- 




/i(si,i) = "1,1 


/l(si,2) = "1,2 


/i(si,b) = Ml,S 


- 


User 1 


xi(«i,i, ai) 


Xi(mi,2,Si,i) 


Xi(mi^S, Sl_B-l) 


xi(l,si^s) 




S2,l 


S2,2 


S2,B 






/l(s2,l) = "2,1 


/l(s2,2) = M2,2 


/i(s2,b) = 1*2, S 




User 2 


X2(M2,l,a2) 


X2(m2,2,S2,i) 


X2{"2,S,S2,B-l) 


X2(l,S2,s) 


Relay 


X3(ai,a2) 


X3(si,l,S2,l) 


X3(si,S-l, S2,s-l) 


X3(si,_b,S2,b) 



TABLE III 

Encoding process for the joint source-channel achievable rate of Thm.[81 



b decodes M2,6), i-C-, the bin indices corresponding to i — 1,2, by looking for a unique pair {ui,U2) such 
that: 

(Sl,fc-1, S2,b_l, Xi(ui, Si^b_l), X2(U2, S2^6-l), X3(si,h_i, S2,fc-l), Vf^^b-l , YS^b) 

G A*J''HSi,S2,X,,X2,X3, W3,Y3). (48) 

The decoded bin indices, ({ti, 7/2), are then given to the relay source decoder Using the bin indices and the side 
information W3 j,, the relay source decoder estimates Si {,,82^;,. More precisely, given the bin indices {ui,U2) the 
relay source decoder declares (si j,, §2,6) as the decoded sequences if it is the unique pair of sequences that satisfies 
/i(si,b) = ■iii,/2(s2,b) = W2 and (si,6, S2,f,, wa^b) e ^2, W^a)- 

Decoding at the destination is done using backward decoding. The destination node waits until the end of channel 
block B + 1. It first tries to decode Si.s,i = 1,2, using the received signal at channel block B + 1 and its side 
information wg. Going backwards from the last channel block to the first, we assume that the destination knows 
(si f,+i, S2.b+i) and consider decoding of (si.b,S2,6). From (si b+i, S2.b+i) the destination can find the bin indices 
(wi^b+i, U2.b+i)- Using this information, its received signal yb+i and the side information w^, the destination 
decodes (si b,S2,6) by looking for a unique pair (§1,82) such that: 

(si,S2,xi(ui,b+i,Si),X2(M2,b+i,S2),X3(si,S2),Wb,yb+i) £ A*^"'> {Si , S2, Xi , X2, X3,W,Y) . (49) 

4) Error probability analysis and corresponding rate constraints: The error probability analysis is given in 
Appendix |D] It is shown that decoding the source sequences at the relay can be done reliably as long as ( 142 al i- 
(I42cl l hold and destination decoding (|49] | can be done reliably if ( I42db - (l42fl l hold. 

Remark 18. In both Thm. |2]and Thm. [8] we used a combination of SW coding and CPM. Since CPM can generally 
allow higher rates, a natural question that arises is whether it is possible to design a scheme based only on CPM, 
namely encode both cooperation (relay) information and the new information using a superposition CPM scheme. 
The problem with this approach in the framework of the current paper, which uses decoding based on joint typicality, 
is that joint typicality does not apply to different blocks of the same RV. For example, we cannot test the joint 
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typicality of s;, and Sb+i, as they belong to different time blocks. Using a CPM only scheme will require us to 
carry such test. For example, using the CPM technique for sending information to both the relay and the destination 
yields the following relay decoding rule: assume that the relay knows (si,b_i, S2,6-i) at the end of block 6—1. 
The relay decodes (si h, S2,6), by looking for a unique pair (§1, §2) such that: 

(si,S2,Xi(si,Si,b_l),X2(s2,S2,b-l),Si,6_l,S2,b-l,X3(si,t_i,S2,6_l),W3,{„y3,fc) G A*J"-\ (50) 

Note that (si, S2) and (si.b_i, 82,6-1) can not be jointly typical since they correspond to different blocks indices, 
i.e. (§1,82) correspond to block b while (si,b_i, 82,6-1) correspond to block 6—1. 

Similarly, the following destination decoding rule becomes: assume that the destination knows (81,6+1,82,6+1) 
at the end of block 6+1. The destination decodes (81,6, 82,6), by looking for a unique pair (si, 82) such that: 

(8l,6+l,82,6+l,Xi(8i,6+l,Si),X2(82,6+l,S2),Si,S2,X3(si,S2),W6,y6+l) E A*^"''> . (51) 

Again, (§1, §2) and (81 6+1, 82,6+1) can not be jointly typical since they correspond to different block indices, i.e. 
(§1,82) correspond to block b while (81.6+1,82,6+1) correspond to block 5+1. 

We conclude that applying the CPM technique for sending information to both the relay and the destination 
can not be used together with a joint typicality decoder It is, of course, possible to construct schemes that use a 
different decoder or carry CPM through intermediate RVs that overcome this difficulty, but these are left for future 
research. 

Remark 19. A comparison of the decoding rules of Thm. [T] with the decoding rules of Thm. [8] reveals a difference 
in side information block indices at the relay and at the destination. The decoding rules of Thm. [T] uses side 
information block with the same index as that of the received vector, while the decoding rules of Thm. [8] uses side 
information block index earlier than that of the received vector. The difference stems from the fact that in Thm. [T] 
the random binning (auxiliary variables) is used for cooperation between the relay and the destination, while in 
Thm. [8] the cooperation is based on the source sequences. In the DF scheme cooperation information is used with 
delay. As it is used with delay when source sequences are used for cooperation the side information block should 
be correlated with the sources blocks. 

Remark 20. Both achievabiUty schemes of Thm. [T] and Thm. [8] use joint typicality decoders. The joint typicality 
decoders can be used only for k = 1 (i.e. n = m). For k ^ 1 Xi and 81 can not be jointly typical. Hence, a 
separation based joint typicality decoders should be used. 

VII. Conclusions 

In this paper we have considered the transmission of arbitrarily correlated sources over MARCs and MABRCs 
with correlated side information at both the relay and the destination. We have first derived an operational separation 
based achievable source-channel rate for MARCs (this rate applies directly to MABRCs as well). This result is 
established by using an irregular encoding scheme for the channel code. We have also showed that for both MABRCs 
and MARCs regular encoding is more restrictive than irregular encoding. 
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Next, we have considered phase and Rayleigh fading MARCs with side information and we have identified 
the conditions under which informational separation is optimal in this setup. Conditions for the optimality of 
informational separation for fading MABRCs have also been obtained. The importance of this result lies in the 
fact that it enables a modular system design (separate design of the source and channel codes) while achieving the 
optimal end-to-end performance. We also note here that this is the first time the optimality of separation is shown 
for the MARC and MABRC configurations. 

Finally, we have considered joint source-channel coding for DM MARCs and MABRCs for source-channel rate 
K = 1. We first showed with an explicit example that joint source-channel codes in general achieve higher rates 
for this scenario compared to separation based codes. We than derived two new joint source-channel achievability 
schemes for DM MARCs and MABRCs. Both schemes use a combination of binning and CPM. While in the first 
scheme joint source-channel coding is used for encoding information to the relay and binning is used for encoding 
information to the destination, in the second scheme binning is used for encoding information to the relay and 
source-channel coding is used for encoding information to the destination. The different combinations of binning 
and source mapping enable flexibility in the system design by choosing one of the two schemes according to the 
quality of the side information and received signals at the relay and the destination. In particular, the first scheme 
has looser decoding constraints at the relay and is therefore better when the source-relay communication is the 
bottleneck, while the second scheme has looser decoding constraints at the destination, and is more suitable to 
scenarios where the source-destination path is more noisy. 

Appendix A 
Error Probability Analysis for Thm.[T] 

Let ei be a positive number such that ei > e, and ei — !■ as e — ^ 0. We start with the relay error probability 
analysis. 

Relay channel decoder: Assuming correct decoding at block 6 — 1, a relay channel decoding error event, E^f^, 
in block b, is the union of the following events: 

El ^ {(xiK,,,<,_i),x2K,,4b-i):Vi(<,_i),v2(4,_o,x3(<,_i,4,_i),y^ ^ 

E2 = {^ui^ul^ : (xi(wi,w^,,_i),X2K,6,4f,_i),viK_f,_i),V2(46-i),X3(<f,_i,M2,b_i),y3,b) e A:^")}, 

-B3 = {3^i2 ■ (xiK,6,<b_i),X2(w2,W2,6-l),Vl(<f,_i),V2(M^_6_l),X3(<b_i,uib_l),y3,b) G 4*^")}, 

El = {3ui ^ u[j„U2 ^ u^^b : 

(xi(ui, X2(u2, "ib-i), vi«f,_i), V2(M^,f,_i), X3K_f,_i, y3,b) G A:^") }. 

Given correct decoding at block 6 — 1, the average probability of channel decoding error at the relay in block b, 
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Pr% B ' is defined by 

nS=p4£^rUu'=2{^;n(i?rr}| <pr{i?r}+^pr{i?ji(i?rr}, (a.i) 

where the inequahty in dA.lt follows from the union bound. From the asymptotic equipartition property (AEP), 
ll36l ch. 5.1], it follows that for sufficiently large n, Pr {£'[} can be bounded by e. 
The event E2 is the union of the following events 

E'2= U ^2(^1), (A.2) 

iie{i,2,...,2"-^i},ui5^«5; 

where i?2(ui) is defined as follows 

M4.b-i),M4.b~i,4.b-i),y3.b) g ^ ui,}. (a.3) 

Let us denote xi(ui, ^_ J by xi, X2(-u^ u^ „_i) by X2, vi{uf ,,_i) by vi, V2(m2,6-i) by V2, X3(uf J 
by X3 and ya by y^. The joint distribution of xi, X2, vi, V2, X3 and obeys 

n 

p(xi, X2, Vi, V2, X3, ys) = Y[ P{y3,j\x2,],X3,j,Vij,V2,j)p{x2,j,Vij,V2j,X3j)p{xij\vij). (A.4) 

Using the chain ( |A4] i in |35, Thm. 14.2.3], Pr can be bounded by 

(ii) 2-«[-f(^i;i'3l^2.Vi,X3]-ei) 5-) 
where (a) follows from the Markov chain V2 — {X2,X3) — {Vi, XijY^). Hence, 

PT{E^,\{Eir}< Yl PT{E^,{n,)\{Eir} 

«ig{i,2,...,2"^i } 

^ 2nlRl-{I{Xv,Y3\X2yi,Xa)-^i)]^ (^^g) 

which can be bounded by e, for large enough n, if 

i?i </(Xi;r3|X2,Fi,X3)-ei. (A.7) 

By following similar arguments to these in (|A.21 i-( |A77] |, we can also show that Pr {£^3 1(£'[)^} can be bounded 
by e, for large enough n, if 

R;<IiX2;Y3\Xi,V2,X3)-ei. (A.8) 

^The overall average probability of channel decoding error at the relay calculated over all B blocks is upper bounded by X]^i^rch^' 
where is the average probability of error at block b assuming correct decoding at block 6—1. We show that P^"^'l^^ is upper bounded 

by a quantity independent of b, denoted P^"Jf^, i.e. P^"^'l^^ < ch' Thus, the average probability of error is upper bounded by B ■ P^"Jf^, 
which goes to zero, for sufficiently large n, for any fixed B. 
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The event is a union of the following events 

El= [J y i?I(ui,U2), (A.9) 

uie{l,2,...,2""i },iii/u!|;_^ U2G{l,2,...,2""2},S2/ti5 

where El{ui,U2) is defined as follows 

El{ui,U2) = {(Xi(ui, U^f X2(W2, W2,6-l)> VlK,f,_i), V2(m2,6-i), yg^fc) 

e A:(")(Xi,X2,Vi,F2,X3,r3)|"i ^<,6,U2 7^t^2,;,}- (A-iO) 

Let us denote xi{ui,uf ,^_-^) by xi, X2(w2, W2,6-i) by X2, vi(iif by vi, V2(w2,b_i) by V2, X3{uf^i,~i^'^2M-i) 
by X3 and y3 b by y3. The joint distribution of xi, X2, vi, V2, X3 and y3 obeys 

n 

p(xi,X2, vi, V2,X3,y3) = ]^p(y3j|wij,t;2j,a;3j)p(wij, W2j,X3j)p(iij,i2j|wi,i,W2j), (A.ll) 

Using the chain dATTT i in ^ Thm. 14.2.3], Pr{£'^(wi,M2)|(i^r)''} can be bounded by 

Pi{El{ui,il2)\iE{y} < 2-«[^(^i.^^-'*'3l^^i,V2,X3)-ei]_ j2) 

Hence, 

Pr{£;i|(£;[)^}< 5^ Pi:{El{u,,u,)\{Eir} 

«ie{l,2,...,2""i J, i2e{l,2,...,2"''2},i2 7^"5_(, 

< 2"[^5:+^5-(^(-^i.-^2;-i'3|Vi,y2,X3)-ei)]^ (A.13) 
which can be bounded by e, for large enough n, if 

+ </(Xi,X2;r3|^^i, V^2, ^3)- ei- (A.14) 

Hence if conditions (|A.7t . ( IA.8I 1 and ( |A.14| i hold, for large enough n, Pj."Jf^ < 4e. 

Relay source decoder: A relay source decoding error event, E^j.^, in block b, is the union of the following 
events: 

i?5^^{(si,6,S2,6,W3,b) ^ A:('")(5i,52,W^3)}, 

E-, ^ {3si ^81,6 : /[(si) = /[(si^b) and (§1, 82,6, W3,6) £ ^2, W^a)}, 

^ {3S2 ^ S2,f, : /2"(i2) - /2"(s2.t) and (si,,, 82, W3,6) e A:('")(5i,52, W^3)}, 
E; ^ {3si ^ 81,6,82 ^ 82,, : - /[(si,,),/2^(§2) = /2"(S2,6) and (ii,i2,W3.b) e A:(™)(5i,52, W^3)}. 

The average probability of source decoding error at the relay in block b, Pr'^rc, is defined by 

P^::l = Pr [e; U 1^^,{E^ n {E;r}\ < Pr + ^ Pr (A15) 

j=5 

where the inequality in dA.lSI l follows from the union bound. From the AEP for sufficiently large n, Pr{i?g} can 
be bounded by e. 
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To bound Pi {E^\{Ej:y} we have 



FT{E;\{E;r}^ Y p(si,S2,W3)x 

(si,S2,W3)eA:<"'(Sl, 52,1^3) 

Pr{3si ^ Si : /[(si) = /r(si),(ii,S2,W3) e A:(")(5i,52, W^3)|si,S2,W3} 



< Y P(S1:S2,W3) 

(si, S2,W3)eA*<'"' (51,52,^3) 



Y Pr{/r(si) = /[(si)|si,S2,W3} 

si#si 

\gieA;('"'(Si|s2,w3) 



(a) 

< 



5^ P(S1,S2,W3)2-"^1 A:(")(5i|s2,W3) 



(si,S2,W3)eA:^'"'(S'l, 52,1^3) 



2-m[R1~H{Si\S2,W3)-ei] 



(A. 16) 



where (a) follows as the bin assignments are independent and uniform; and (6) follows from 11351 Thm. 14.2.2]. 
The RHS of inequality ( IA.I6I 1 can be bounded by e, for large enough m, if 

H{S,\S2,W3) + e,<R';. (A.17) 

Following similar arguments as in ( IA.16b . we can also show that Pr {E^\{EJ!^Y} can be bounded by e, for large 
enough ni, if 

H{S2\Si,W3) + ei<Rl (A.18) 
and Pr {E^KE^y} can be bounded by e, for large enough m, if 

H{Si,S2\W3) + ei < Rl + R^. (A.19) 

Hence, if conditions ( IA.17b . ( IA.18I ) and ( IA.19b hold, for large enough m, Pr™rc < 4e. Combining conditions 
(IA.7b . (IA.8I ) and (IA.14I ) with conditions ( IA.17b . (lA.lSb and ( IA.19b and using ^[ = i = 1, 2 yields the relay 
decoding constrains dSab-dScll in Thm. [T] for the achievability of rate k. 



Next the destination error probability analysis is derived. 

Destination channel decoder: Assuming correct decoding at block 6 + 1, a destination channel decoding error 
event, E^^, in block b, is the union of the following events: 

E^ ^ {(xiK,+i,<,),x2K,+i,4,),vi(<,),V2(4j,X3(<,,4j,y6+i) i 

Ei = ^ ufj^ ; (xiK j,+i,ui),X2(M5^6+i."2,J>vi(ui),V2(u^^J,X3(wi,M^ j,),yf,+i) e 

Ei = {3U2 + : (xiKf,+ i,<f,),X2K^b+i,M2),ViK^J,V2(M2),X3K,,,U2),y6+l) G A*^"'>} , 

Ei ^ {3ui ^ uf ,^,U2 ^ U2,b : (xi(<,6+i,ui),X2(w2,6+i:"2),vi(ui),V2(u2),X3({ti,M2),yb+i) e 
The average probability of channel decoding error at the destination in block 6, J^, is defined by 

Ptck = 1^1 U "^Ui^f n )=}) < Pr {Et} + ^ Pr {iJj'Ki?? )^}, (A.20) 

J j=2 
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where the inequahty in ( IA.20I ) follows from the union bound. From the AEP it follows that for sufficiently large 
Pr [Ef] can be bounded by e. The event E2 is a union of the following events 

Ei= U E^iui) (A.21) 

«ie{i,2,...,2"-"S'},iii5^M5',, 

where is defined as follows 

■E^2 (Wl) - {(xiK.b+i, Wl), X2(u2,b+1, ""2,,,), Vi(ui), V2(U2^J, X3(ui, ,,), Yb+i) 

e4*(")(Xi,X2,i^i,t/2,X3,r)|«i (A.22) 

Using im Thm. 14.2.3], {Ej{ui)\{EfY] can be bounded by 

(ii) 2-«[-f(^i,^3;i'l^2,V2)-ei] 23) 
where (a) follows from the Markov chain Vi — (XijX^) — Y. Hence, 

Pr{E^\iEtr}< Pr {E^{ui)\{Etr} 

< 2nlRl-{I{Xi,Xr,Y\X2y2)-ei)]^ (-^ 24) 

which can be bounded by e, for large enough n, if 

Ri<IiX,,X3;Y\X2,V2)-e,. (A.25) 

Following similar arguments as in (IA.21l i-( IA.25l l, we can also show that Pr {Ei\{Efy} can be bounded by e, for 
large enough n, if 

Rl<I{X2,X3;Y\Xi,Vi)-ei, (A.26) 

and Pr {Ef\{EfY^ can be bounded by e, for large enough n, if 

Rf + R^<IiXi,X2,X3;Y3)~ei. (A.27) 

Hence, if conditions jA.25l l, jA.26l l and (IA.27b hold, for large enough n, -Pj")j < 4e. 

Destination source decoder: The destination source decoding error probability analysis is similar to the relay 
source decoding error probability analysis. The average probability of source decoder error at the destination in 
block b (defined in the same manner as in the relay) can be bounded by e, for sufficiently large m, if 

H{Si\S2,W)+ei<Rf, (A.28a) 

H{S2\Si,W)+ei<Ri (A.28b) 

H{Si,S2\W)+ei<Rf + Ri (A.28c) 

Combining conditions (IA.25I I. (IA.26I I and ( IA.27I I with conditions (IA.28b and using Rf = iRfj = 1,2 yields 
the destination decoding constrains dSdli-dSft in Thm. [T] for the achievability of rate k. 
This concludes the error probability analysis for Thm. [T] 
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Appendix B 
Proof of Thm.[3] 

Let Fe ^> as n,m oo, tor a sequence or encoders and decoders , /2 1/3 , 9 : 9^ > 

such that K — n/m is fixed. We will use Fano's inequality which states 

HiS"^, S^lS'^s, < 1 + mPi"'") log \Si X 

^m5(Pi'"'")), (B.l) 

where 6{x) is a non-negative function that approaches ^ as x ^ 0. We also obtain 

= 5^1^?, W3") (B.2b) 

>H{Sr\Y3^,W^,S^), (B.2c) 

where ( |B.2a| l follows from the fact that conditioning reduces entropy; ( |B.2bt follow from the fact that {S™^, S^^) 
is a deterministic function of Yg" and VF™; and (|B.2c| i follows from non-negativity of the entropy function for 
discrete sources. Constraint ( I14ab is a consequence of the following chain of inequalities: 



k=l k=l 

n 

= ^ ^H{Y3^k\X2,k, X^^k 
fc=l 
n 



H{Y3^k\Xi^k, X2,k, X^^k) 



(B.3a) 



k=l 



- H{Y,,k\Sr, 5^, Wr,W'^, ^ti, Xl,,Xl,,Y,y,Y'^-') 

n 

> J2 [H{Ys.k\ST,WS',Y,',l\X2M,X3^k) 



fe=i 



H{Y3^k\Sr,ST,Wr,W"',Y,y) 



n 



fe=i 



(B.3b) 



(B.3c) 



(B.3d) 



HSr, W"^- Y^^k\Y^-\SlT, W^) 



k=l 



= /(ST , W^" ; ^3" I S^" , W3" ) 

= //(snsT, w^3") - //(sriKT, 5*2", w^) 

>miJ(5i|52,W^3)-m<5(Pi™'")), 



(B.3e) 
(B.3f) 

(B.3g) 
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where (|B.3a| l follows from the memoryless channel assumption; ( |B.3bl l follows from the Markov relation 
(S'f ,S'^\ W3™, W^") - {X^j^,Xl^,Xl^,Y^J^,Y''-^) - Ys^k (see [40]); (lB3cb follows from the fact that condi- 
tioning reduces entropy; (IB.3db follows from the fact that X2.k is a deterministic function of S™, and ^ is a 
deterministic function of (W^ ,Y^^^); ( IB.3el i follows from the chain rule for mutual information; (IB.3fb follows 



from the non-negativity of the mutual information; and (B.3gi follows from the memoryless sources and side 
information assumption and from equations ( IB.lb - (IB.2l l. 

Following arguments similar to those that led to ( |B.3g| i we can also show that 

n 

^/(^2,fc;>3.fc|^i,fc,^3,fc) > mH{S2\Si,W3)~mS{P^"''''^) (B.4a) 

k=l 



J2HXi,k,X2X,Y3,k\X3,k) > mHiSi,S2\W3)-mSiPj;"'-"'^). (B.4b) 

k=l 

From ( IB.3b -( IB.4l ). we have that if k is achievable for the MABRC then 

1 " 1 

- V/(Xi,fe;r3,fe|^2.fc,^3.fc) > -{H{Si\S2,W3)~e), (B.5a) 



n 

fe=l 



1 " 1 

-yi{X2,k;Y3,k\Xi,k,X3,k) > -{H{S2\Si,W3)-e), (B.5b) 

k=l 

1 " 1 

-y2l{Xi,k,X2,k;Y3,k\X3.k) > -{H{Si,S2\W3)-e). (B.5c) 



n 

k=l 



for any e > and large enough n. Using the concavity of mutual information over the set of all joint distributions 
p{xi,X2, X3), taking the limit as m, n — 00 and letting Q leads to conditions (fT4l l. 
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Appendix C 
Error Probability Analysis for Thm.|7] 

We start with the relay error probability analysis. The average decoding error probability at the relay in block b, 
Pr"\ is defined by 

p{n) A p(si.f,, S2,6) Pr {relay error|(si,b, S2,6) are the source outputs} 

< ^ p(si,6,S2,fc) + ^ p(si,b, S2,h)Pr {relay eiTor|(si, {,,82,6) are the source outputs}(C. la) 

(si,6,S2,b)^A;'"'(S'l,S2) (si.b,S2.b)eA;*"'(Sl.S'2) 

<e+ X! P(si.'"S2,h) ^ p(w3,{,|si,fc,S2,{,) X (C.lb) 

(si,b,S2,b)eA:(")(5l,52) W3,beW3" 

Pr {relay error|(si S2,{,) are the source outputs, W3 {, is the side information} 

<e+ ^ p(si,h,S2,h) ^ p(w3,{,|si,h,S2,{,) 

(si,b,S2,b)eA:<">(Sl,S2) W3,b^A*<"'(M/3|si,b,S2.b) 

+ ^ p(si,{„S2,fc) ^ p(w3,fc|si,fc,S2,fc) X 

(si.b,S2,b)eA:*"'(S'l,52) W3,bGA:<"'(iy3|si,b,S2,b) 

Pr {relay error|(si,{,, S2,b) are the source outputs, W3,b is the side information} (C.lc) 

<2e+ ^ p(si,{„S2,6,W3,6) X 

(si.b,S2,b,W3,b)eA:<"'(Si,S2,W3) 

Pr {relay error|(si,{,, S2,h) are the source outputs, W3,h is the side information}, (C.ld) 

where ( IC.lal i. ( IC.lcl i follows from the union bound and (IC.lbb . ( IC.ldl ) follows from the AEP, for sufficiently large 
n. 

In the following we show that for (si,b, S2,{,, W3 {,) S ^*''"'' (iSi, S'2, W3), the summation terms in dC.ldl l can be 
upper bounded independently of (si.fc, S2,{,, W3,{,). In order to show the above we assume that (si h, S2,{,, W3,{,) G 
Ae''"''(iS'i, 5*2, W3) and let ^ denote the event that the triplet (si,{,, 82,6, W3,fc) is the sources outputs and the side 
information at the relay. 

Let eq be a positive number such that eo > e and eg — !" as e — ^ 0. Assuming correct decoding at block 5—1 
(hence (ui,{,_i, U2,&-i) are available at the relay), a relay decoding error event, E^, in block b, is the union of the 
following events: 

E'l ^ {(Si,h,S2,{„Xi(Si,{„Mi,t_i),X2(s2,t,U2,{,-l),Vi(Mi,6_i),V2(M2,h-l),X3(wi,{,_i,-U2,6-l),W3,{,,y3,{,) ^ A*^"^}, 

El = {3(81,82} ^ (8i,{„S2,{,) : 

(Si, 82, Xi(si, Ui,f,_i), X2(S2, -U2,{,_l), Vi('Ui,{,_l), V2(u2,b_l), X3 (ui,{,_i , M2,6-l ) , ^3,b, Y^.b) £ A**^"^ } . 

From the AEP, for sufficiently large n, Pr{£'[|f^} can be bounded by e. Therefore by the union bound 

Pr{£;n^} <e + VT{El\2!,{E{Y]. (C.2) 
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The event E2 is the union of the following events: 

E^, ^ {3si / si,6 : 

(Si,S2,6,Xi(Si,'Ui^,,_l),X2(s2,b,-U2,6_i), Vi('Ui^,,_l), V2(-U2,6_i),X3(ui^;,_l,M2,b-l), W3^b,y3^b) G A*^"^}, 
i?2"2 = {3s2 7^82,6 1 

(Si^f,,S2,Xi(Si^f„Ui^f,_l),X2(s2,W2,6-l), Vi(wi,&_l), V2(w2,6-l),X3(ui,fc_i,U2,(,_l), W3_,„y3_f,) G A*^"^}, 
i;23 - {3S1 7^ Si,6,S2 7^ S2,6 : 

(si,S2,Xi(si,Ui^b_.l),X2(s2,U2,b-l), Vi(mi,6_i), V2(w2,6-l),X3(wi,6_i,M2,6-l),W3,f„y3^f,) G AJ^"^}. 

Hence, by the union bound 

3 
i=i 

To bound Pr {E^i\&, we define the event E^^{si) as follows 

^21 (§1) - {(si,S2,6,Xi(si,Ui^b_i),X2(S2,6,U2,b-l), Vi(mi,6_i), V2(m2,6-i), 

X3(wi,b-1, W2,b-l), W3,f„ y3,b) G 4*^") 52, Xi,X2, VuV2,X3, W3,Y3)\~Si ^ Si,b}. (C.4) 

Let us denote 82,6 by S2, xi(si, ui^b_i) by xi, X2(s2,b, '"2,6-1) by X2, vi(Mi,b_i) by vi, V2(M2,b-i) by V2, 
X3(iti,b_i, it2,6--i) by X3, W3,b by W3 and y3_b by y3. Note that the joint distribution of Si, S2, xi, X2, vi, V2, 
X3, W3 and y3 obeys 

p(si,S2,Xl,X2, Vi, V2,X3, W3,y3) 

n 

= Y[p(^^^o)Pi^2j,W3j)p{x2j,y3,j\s2j,W3j,Vij,V2J,X3j)x 

i=i 

p{vij,V2J,X3^j \sij,S2,j,W3^j)p{xij\si^.j,Vij,V2,j,X3j). (C.5) 

Equation (IC.5l l shows that the assumption of |9 Lemma, Appendix A] on the conditional distribution is satisfied. 
Hence, we can use this lemma to bound Pr |£'2]^(si)|i^, with the following assignments: 

Zi = (S2, W3), Z2 = Si, Z3 = (X3, Vi, V2), Z4 = Xi, Z5 = (X2, Y3), (C.6) 

as follows 

2-ri[I{XuY3\S2,X2,Vi,X3,W3)->io] 7) 
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where (a) follows from the Markov chain V2 — (S'2, -'^2, VijX^, W3) — Y3. Hence 

pr{E;,\^,iEir}< pr{i?2^(§i)i^,(i?rr} 

SieA*<"'(Sl|s2,b,W3,b) 
< ^ 2""[^(^l''^3|S'2,X2,yi,X3,W3)-eo] 

SlGA;'"'(Sl|s2.6,W3,b) 



< 



2-«[/(Xi;Y;,|S2,^2:Vi:X3,W3)-eo] 



A:(")(^l|s2^b,W3,b) 

^ 2n[H{Si\S2,W3)-IiXi;Y3\S2.X2,Vi,X3,W3)+2eo] (^q g-) 

which can be bounded by e, for large enough n, if 

^f(5i|52,VF3) </(Xi;r3|^2,X2,l^i,X3,W^3)-2eo, (C.9) 

Following similar arguments as in ( |C.4| )-( |C8] l. we can also show that Pr {E^2\^, (ElY} can be bounded by e, 
for large enough n, if 

H{S2\Si,W3) < I{X2;Y3\Si,Xi,V2,X3,W3) -2eo, (C.IO) 
and Pr {i^jal^^, {ElY} can be bounded by e, for large enough n, if 

H{Si,S2\W3) < IiXi,X2;Y3\Vi,V2,X3,W3) -2ea. (C.ll) 



Hence, if conditions (I40al i-( l40cl i hold, for large enough n, 

3 

Pr{i?2-|^,(£;[)^} <Y,PHE-2j\^,{Eir} < 3e. (C.12) 

Combining equations jC.ll l. (IC.2l i and ( IC.12b yields 

< Pr{£;j|^, (i;^^} +3e < 6e. (C.13) 
Next the destination error probability analysis is derived. 

Destination channel decoder: Let (si.b+i, S2,b+i) be the output of the source in block 6 + 1. Assuming correct 
decoding at block b + 1, a destination channel decoding error event, E^f^, in block b, is the union of the following 
events: 

Ef = {(si,b+i,S2,b+l,Xi(si,f,+ i,'Ui,b),X2(s2,&+l,U2,(,),Vi(Mi_b),V2(u2,f,),X3(ui,(,,M2,&),Wb+i,yb+i) ^ yl*^")}, 

E!^ ^ {Bill ^ : 

(si,6+l,S2,b+l,Xi(si,b+l,{tl),X2(s2,b+l,U2,6), Vi(wi), V2(m2,&),X3(wi,M2,6), Wf,+ i,yf,+ i) G A*^"'^}, 

El ^{3u2^ U2.b ■■ 

(Sl,6+l, S2,b+i, Xi(si,b+1, Ul,;,), X2(S2,(,+1, {t2), Vi('Ui,b), V2(u2), X3(wi,6, U2), Wf,+i, y^+i) G AJ^") }, 



Ef = {3ui / Ui^b, U2 ^ U2. 



b ■ 



(si,f,+i, S2,6+i, Xi(si,6+i, ui), X2(s2,6+i, 122), Vi(ui), V2(ii2), :x-3{ui, U2) , v/b+1 , Yb+i) e }. 
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The average probability of channel decoding error at the destination in block 6, Pji]}f^, is then given by 

^i"ci = Pr {Ef U U^=2Ef} < PT{Et} + J2 Pr {Ef\{Efr}, (C.14) 

where the inequality in ( |C.14| i follows from the union bound. From the AEP for sufficiently large n, Pr{Ef} can 
be bounded by e. The event i?| is the union of the following events 

E^^ U (C.15) 

where i?|({ti) is defined as follows 

E^iui) = {(si,h+i,S2,h+l,Xi(si,b+i,Ui),X2(s2,fc+l,M2,6), Vi(wi),V2(m2, 6),X3 ({41,-^2, b),Wb+i,yf,+ i) 

e 52, Xi,X2,Vi,V2,X3, W, Y)\ui^ui^b}. (C.16) 

Let us denote Si^b+i by Si, S2,b+i by S2, xi(si^b+i, mi) by xi, X2(s2,6+i, U2,b) by X2, vi(ui) by vi, V2(u2,b) by 
V2, X3(mi, U2.b) by X3, Wb+i by w and y^+i by y. The joint distribution of Si, S2, xi, X2, Vi, V2, X3, w and y 
obeys 

p(si, S2, Xl, X2, Vi, V2, X3, w, y) 

n 

= n P^yj I ^^'j ' ' 2^2 J , V2,j ,Wj)x 

p{xij,X3j,Vi^j\si^j,S2,j,X2^j,V2,j,Wj)p{sij,S2,j,X2,j,V2,j,Wj). (C.17) 

Using the chain dCTTI i in ^ Thm. 14.2.3], Pr {El{ui)\{EfY} can be bounded by 

Pt {E^{ui)\{E^y} < 2-"[-f(^i'^3'^i'^l'S'i^'S2-^2,V2,w)-eo] 

where (a) follows from the Markov chains Vi - {Xi, X2, X3, Si,V2,W) - Y and (^2, W^) - 1^2,^2) - Y. 
Hence, 

PT{E^\iEtr}< J2 PT{E^iui)\iEfr} 

iiie{l,2,...,2"-"i},ui5^Mi,i, 

2"[-Ri-(-f(-^i.^3:i'|Si,X2,V2)-eo)] 19) 

which can be bounded by e, for large enough n, if 

Ri < I{XuX3;Y\Si,X2,V2) ~ eo. (C.20) 

Following similar arguments as in (IC.15b -( IC.20b . we can also show that Pr {El\{EfY} can be bounded by e, 
for large enough n, if 

R2 < I{X2,X3;Y\S2,Xi,Vi) - CO, (C.21) 
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and Pr {EfKEfY] can be bounded by e, for large enough n, if 

Ri + R2 < I{Xi,X2,X3;Y\Si,S2) - eo. (C.22) 
Hence, if conditions (IC20l l. (IClTT l and (IC22T l hold, for large enough n, P^"j^ < 4e. 

Destination source decoder: The error probability analysis for decoding the sources at the destination is similar 
to the relay source decoder error probability analysis done for the separation-based achievable rate detailed in 
Section [nil From this analysis it follows that the average probability of source decoding error at the destination in 
block b (defined in the same manner as in the relay, in Section HIll i can be made arbitrarily small, for sufficiently 
large n, if 

H{Si\S2,W)+eo<Ri, (C.23a) 

H{S2\Si,W)+eo<R2, (C.23b) 

H{Si,S2\W)+eo<Ri+R2. (C.23c) 

Combining conditions iC.lQi . ( IC.21b and (IC.22b with conditions ( IC.23I I yields the destination decoding constrains 
(I40dl i-(l40fb in Thm. |7] for the achievability of rate k = 1. 

Appendix D 
Error Probability Analysis for Thm. [8] 

We start with the relay error probability analysis. 

Relay channel decoder: Assuming correct decoding at block 6 — 1, a relay channel decoding error event, £'J^, 
at block b, is the union of the following events: 

E'l = {(si,b_i,S2,6-l,Xi(ui,f,,Si_f,_i),X2(w2,6,S2,b-l),X3(si,6„l,S2,6_l),W3^b_l,yb) ^ A*J'^^}, 

E2 ^ {3wi 7^ Wl,6 : (si,6_i,S2.b-l,Xi({ti,f„Si^b-l),X2(w2,6,S2,b-l),X3(si^b_l,S2,b-l),W3,6_l,yb) £ A*'"^}, 

El = {3U2 ^ U2,6 : (si,6-l,S2,b-l,Xi(Mi,f„Si^b-l),X2(M2,6,S2,b-l),X3(si,b-l,S2,b-l),W3,b_l,yb) £ A**^"^}, 

E'l = {3ui ^ Ul,b,it2 ^ W2,6 : (Sl,6_l, S2,b-1, Xi(-Ui,6, Si^b-l), X2(m2,6, S2,b-l), X3(si^b-1, S2,b-l), W3_f,_i, yb) G A* 

~ in] 

The average probability of channel decoding error at the relay in block b, Jj^, is defined by 

4 

Pr% - ^^{E{ U "^-^E^) ^ {^n + E ^^{E]mr}, (D.i) 

where the inequality in dP.lt follows from the union bound. From the AEP for sufficiently large n, Vi {E\} can 
be bounded by e. Let eo be a positive number such that eo > e and eo ^ as e — ^ 0. 
The event £'2 is the union of the following events 

El^ U ^2(^1)' (D.2) 

ilie{l,2,...,2"-"i },tii7^«i,6 
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where i?2(ui) is defined as follows 

: { (Si,fc_i, S2,6-l, Xi({ti, Si,b_l), X2(u2,b, S2,6_l), X3(si,f,_i, S2,6-l), W3,6_i, ya^b) 

G A:(")(5l,52,^l,^2,^3,W^3,i^)|wi 7^"l,b}. (D.3) 

Let us denote Si,6_i by Si, 82,6-1 by S2, Xi({ti, Si,6_i) by Xi, X2(w2,6, S2,b-i) by X2, X3(si,6_i, 82,6-1) by X3, 
W3,6-i by W3 and ya ^ by ya. The joint distribution of Si, S2, xi, X2, xa, wa and ya obeys 

p(si,82,Xi,X2,xa,W3,y3) 

n 

= n Piy3,]\si,],S2,j,X2,j,X3j,W3j)p{xij\sij,S2,j,X2,j,X3j,W3j)p{sij,S2,j,X2,j,X3j,W3j). (DA) 

Using the chain (Ell in |[35^, Thm. 14.2.3], Pt {E^{ui)\{Eiy} can be bounded by 

Pi- {E^{UI)\{EIY} < 2-"[^(^l'^3|Sl,S2,X2,X3,W3)-eo] 

2-ri[I{Xi-Y3\SuX2.X3)-eo] (-j) 5) 

where (a) follows from the Markov chain (5*2, M^a) ~ {Si, X2, X3) — Y3. Hence, 

Pr{E;\{E{r}< PriE^^iu^mir} 

< 2nlB.i~iIiXi;Y3\Si,X2,X3)-eo)]^ ^ 

which can be bounded by e, for large enough n, if 

Ri<I{Xi;Y3\Si,X2,X3)-eo. (D.7) 

Following similar arguments as in ( |D.2| i-( |DTt . it follows that Pr {£'3 1(£'[)^} can be bounded by e, for large 
enough n, if 

R2 < I{X2;Y3\S2,Xi,X3) - eo, (D.8) 
and Pt {El\{Eiy} can be bounded by e, for large enough n, if 

Ri+ R2 < IiXi,X2;Y3\Si,S2,X3) ~ eo. (D.9) 

We conclude that when conditions (ID.7l i. ( ID.8I 1 and iD.9\ hold, for large enough n, P^^X — 

Relay source decoder: The relay source decoder error probability analysis is similar to the error probability 
analysis of the relay source decoder in the separation-based achievable rate detailed in Section |III] The average 
probability of source decoder error at the relay can be made arbitrarily small, for sufficiently large n, if 

H{Si\S2,W3) + €o<Ri, (D.lOa) 
ff(52|5i,VKa) + eo<i?2, (D.lOb) 
H{Si,S2\W3) + eo<Rx + R2. (D.lOc) 
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Combining conditions ( ID.7I ). ( ID.8I ) and iD.9i with conditions dP.lOl l yields the relay decoding constrains ( 142 a| l- 
( I42cl ) in Thm. [8] for the achievability of rate k = 1. 

Next the destination error probability analysis is derived. 

Destination error probability: The average decoding error probability at the destination in block 6, Pj"'', is 
defined by 

-^i"^ ^ 5Z p(si,fc, S2,b) Pi' {destination eiTor|(si S2_b) are the source outputs} 




where (ID.lll i follows arguments similar to those led to ( IC.ldb . 

In the following we show that for (si.f,, S2.b, Wf,) G A^^^\Si, S2,W), the summation terms in ( ID. 1 11 1 can be 
upper bounded independently of (si.b, S2,6, w;,). In order to show the above we assume that (si.b, S2,6, Wb) e 
A*'"^^\Si, S2,W) and let ^ denote the event that the triplet (si.b, S2,6, Wb) is the sources outputs and the side 
information at the destination. 

Let eo be a positive number such that ep > e and eq ~^ as e — > 0. Assuming correct decoding at block 6 + 1 
(hence (wi b+i, U2.b+i) are available at the destination), a destination decoding error event, i?*^, in block b, is the 
union of the following events: 

El = {(si,b,S2.f„Xi(ui,b+l,Sl,6),X2(M2,6+l,S2,b),X3(si,b,S2^b),Wb,yb+i) ^ A*^"^}, 




(si,b.S2,b)e5I'x5J 




(si,(,,S2,b,Wi,)G 



E^ = {3(si,S2) ^ (si,b,S2^b) : (si,S2,Xi(ui^b+i,Si),X2(M2,6+i,S2),X3(si,S2),Wb,yb+i) e 



From the AEP, for sufficiently large n, Pr |^} can be bounded by e. Therefore by the union bound 



Pr {E'^IS'} < e + V^{E^\&, [Eff] . 



(D.12) 



The event E2 is the union of the following events: 

E21 = {3si 7^ si,b : (si,S2,b,Xi(-ui,b+i,Si),X2(M2,6+i,S2,b),X3(si,S2,6),Wb,yb+i) e 
E22 = {3s2 ^ S2,6 : (si^b,S2,Xi(wi,6+i,Si^b),X2(M2,b+i,S2),X3(si,b,S2),Wb,yb+i) e 

- {3si i= Si, 6, §2 7^ S2,b : (si,S2,Xi(ui,b+i,Si),X2(w2,6+i,S2),X3(si,S2),Wb,yb+i) e 



Hence, by the union bound 



3 




(D.13) 



(D.14) 
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Let us denote S2,6 by S2, Xi (ui^b+i, §1) by xi, X2(u2.&+i, S2,fc) by X2, X3(si,fc, 82,6) by X3, Wh by w and y^+i by 
y. Note that the joint distribution of Si, S2, Xi, X2, X3, w and y obeys 

n 

p(si , S2 , Xi , X2 , X3 , W, y) = Jl p(si, J , S2,j , )P{X2,J , Vj I S2,j , Wj )p{xij , Xs^j \S1J ,S2,j,Wj). (D. 1 5) 

Equation ( ID. 151 ) shows that the assumption of the Lemma |9 Lemma, Appendix A]B|on the conditional distribution 
is satisfied. Hence, we can use this lemma to bound Pr {ii'|]^(si)|i^, {E^y^ with the following assignments: 

zi = (S2,w),z2 = ii,Z3 = 0,Z4 = (Xi,X3),Z5 = (X2,Y), (D.16) 



as follows 



Hence 



Pr{i;^i(si)|^,(£;f)=} <2^"[^(^i'^3''^l^^'^^'^)-<^°l. (D.17) 



si/si 

glGA'(")(Sl|s2.6,Wi,) 



X3;Y\S2,X2,W)-eo] 



iieA;(")(Si|s2,6,Wi,) 



< 



2-n[I{XuX3-Y\S2,X2,W)-eo] 



A:^"\Sl\s2,b,Wb) 

2"[ff(SilS2,W')-/(Xi,X3;y|S2,X2,W)+2£o] jg-) 

which can be bounded by e, for large enough n, if 

HiSi\S2.W) < I{Xi,X3;Y\S2,X2,W) -2eo, (D.19) 

Following arguments similar to those that led to (ID. 18b . we can also show that Pr {E^^I^, {EfY} can be 
bounded by e, for large enough n, if 

HiS2\SuW) < I{X2,X3;Y\SuXi,W) -2eo, (D.20) 

and Pr{E^^\&,{EfY} can be bounded by e, for large enough n, if 

H{Si,S2\W) < I{Xi,X2,X3;Y\W) -2eo, (D.21) 



Hence, if conditions (I42al l-( l42cb hold, for large enough n, 

3 

PilE^l^AEtY} <YP'^{E2j\^AEty} < 3e. (D.22) 

^The following Lemma is a slight variation of f^" Lemma, Appendix A]. Lemma: Let {Zi, Z2, Z3, Z^, Zs) be random 
variables with joint distribution p(zi, 2:2, Z3, 24, 2:5 ). Fix (zi.za) e A:<"^(Zi,Z2), and let Z3, 24,25, be drawn 
according to Pr {23 = Z3, 24 = Z4, Z5 = Z3 |zi , Z2} = Yl"^iPizs,j\z2,j , Zi,j)p{z4j\z2,j , Zij)p{z5j\z3j , zij). Then 
Pr|(zi,Z2,23,24,Z5) G A*'"'(Zi,Z2,Z3,Z4,Z5)} < 2-'^i"-^3-:Zi\Zi,Z2)+l(Zs;Z2,Zi\Zi.Z3)-S€]^ The proof follows arguments 
similar to the proof of (9] Lemma Appendix A]. 
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Combining equations dP.llI ). ( |D.12t and ( ID.22I ) yields 

P]"^ < FT{E^\&,{Efy} +3e < 6e. (D.23) 
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